Open Osndok opened 3 years ago
This was one of the harder parts of making OpenBSD work well in QubesOS. OpenBSD doesn’t expose Xen grants to userspace, so porting the QubesOS tools would require a custom kernel build. I managed to find most of the information needed in XenStore, but I was not able to find the netmask or gateway, so I wound up using static ARP entries and a fake gateway instead. This was made even more difficult because of an OpenBSD kernel bug which prevented the obvious approach from working. Patches sent to fix it were ignored.
If the VM lacks PV drivers (i.e. uses emulated ethernet instead of PV one), then it will get network configuration via DHCP. This DHCP server is running within stubdomain providing the emulated device and as such is not available for PV devices. To have DHCP server for PV devices too, it would need to run in network-providing VM directly, which would add an attack surface (bugs in dnsmasq are discovered from time to time). Some solution to get DHCP could be having a minimal one, preferably in a memory-safe language, and heavily sandboxed (at least namespaces + seccomp-bpf). IMO if working out of the box is the priority, it should be DHCP. But note also, having DHCP client running also adds an attack surface (on the downstream VM this time), but this depends on whether that client is active, regardless of whether anything responds to its requests.
An alternative solution indeed could be something compatible with cloud-init (although requiring cloud-init installed makes it less out of the box solution). Looking at the supported data sources, the most fitting would be NoCloud - i.e. an extra disk with a configuration file.
I managed to find most of the information needed in XenStore
Specifically:
be_path=$(xenstore-read device/vif/0/backend)
ip=$(xenstore-read "$be_path/ip")
mac=$(xenstore-read "$be_path/mac")
Oh, wow! Two excellent finds.
I could see the NoCloud
option possibly fit well with a larger strategy for qubes-air-ish things (such as if a controller qube could request a disposable qube to be spun up, giving it some arguments in the form of a cloud-init blob).... making a qubes install a mini cloud provider.
I also agree with your comments concerning DHCP
, in principal, but I suppose I was just looking for the shortest & easiest-to-implement path (which if it is just patching some default xen scripts to add dns, gateway, and netmask... might be easy?).
One thing to note, is that if a machine image has xenstore libraries, it likely has the PV kernel modules too... so one might see giving this data via xenstore as a means to climb out of "falling into the hole" of having the PV drivers...
Also, this might not be a qubes-specific problem (Xen DHCP), as I see instructions out on the internet like this:
Clear the current XenStore data values:
Set-XenVM -VM $VPXVM -XenstoreData $null
Store the NetScaler Gateway IP address, default gateway and subnet mask addresses in an object:
$Dictionary= New-Object 'system.collections.generic.dictionary[string,string]'
$Dictionary.Add("vm-data/ip","<NetScaler IP>")
$Dictionary.Add("vm-data/netmask","<Subnet Mask>")
$Dictionary.Add("vm-data/gateway","<Gateway Address”)
...and here https://github.com/krobertson/xenserver-automater
UUID=`xe vm-install template=mytemplate new-name-label=newvm`
xe vm-param-set uuid=$UUID xenstore-data:vm-data/ip=192.168.1.20
xe vm-param-set uuid=$UUID xenstore-data:vm-data/gw=192.168.1.254
xe vm-param-set uuid=$UUID xenstore-data:vm-data/nm=255.255.255.0
xe vm-param-set uuid=$UUID xenstore-data:vm-data/ns=192.168.1.254
xe vm-param-set uuid=$UUID xenstore-data:vm-data/dm=mydomain.com
xe vm-start uuid=$UUID
...or here http://xoa.io/deploy
xe vm-param-set uuid=$uuid xenstore-data:vm-data/ip=$ip xenstore-data:vm-data/netmask=$netmask xenstore-data:vm-data/gateway=$gateway xenstore-data:vm-data/dns=$dns
The problem you're addressing (if any)
As mentioned in the qubes documentation, modern linux kernels are unable to make use of the DHCP server to configure their networking.
When setting up one-off guests, it may not be too difficult for experienced users to manually fetch & copy the details from dom0, but it would be a much better user experience if this could be done automatically from within the guest through means such as cloud-init, jeos-firstboot, or even custom-written scripts.
At present, it seems the only way to acquire this information from the guest would require communication over the qubes agent channel, which would effectively require qubes binaries, which others would understandably be reticent to package & maintain for such a niche.
On the other hand, if the network addresses were made available available via
xenstore
, then this could be accomplished by just about any OS that supports living under xen, because they would all have the xenstore capability.Describe the solution you'd like
I would like to see the network addresses (ip, netmask, gateway, dns) made available via
xenstore
by any means that the maintainers of qubes see fit.It would be swell if it could be done in a way that a widely-supported provisioning mechanism would already recognize (such as that
xenstore-read vm-data
json blob), or created as an entirely fresh qubes specification (such asxenstore-read qubes/net/gateway
).In the first case, it would "just work"... which would be terrific.
In the second case, it might only require the user to create a script or two such as:
Where is the value to a user, and who might that user be?
To the advanced qubes users, this would mean that they could create standalone HVMs that could be successfully cloned in an automatic fashion, or simply to avoid the nuisance of manually tweaking the network configuration each time.
To the maintainers of the provisioning mechanisms, it would be far more agreeable (and easy) to add an extra code path to support qubes if it only required testing for the presence of a xenstore variable, than to package, maintain, and bake-in the qubes agent/db binaries.
To the less-advanced qubes users, it might mean that more ISOs would "just work" going forward into the future.
To the qubes maintainers & community, it might mean marginally fewer "networking doesn't work" support requests, or a lessened "qubes is too hard" disposition.
Describe alternatives you've considered
Additional context
This came to light as I spent way too many hours trying to get an OS I was unfamiliar with (SUSE JeOS; since it has a xen image) to work as a qubes guest, 99% of the time was spent trying to manage the quirks of manually configuring networking without any of the usual nicities (being a super-lean distro).
Relevant documentation you've consulted
Related, non-duplicate issues
5605
6145