search

rancher / os

Tiny Linux distro that runs the entire OS as Docker containers
https://rancher.com/docs/os/v1.x/en/
Apache License 2.0
6.44k stars 657 forks source link

Include XenServer guest utilities #813

Open ExecDevOps opened 8 years ago

ExecDevOps commented 8 years ago

Hi there!

To make RancherOS play like a champ on XenServer (i.e. VM motion between hosts, manageability, performance monitoring, etc) it would need to include the Xen guest-utilities (https://github.com/xenserver/xe-guest-utilities).

XenServer has started to incorporate support for managing Docker containers in VM's and there is already support for CoreOS. CoreOS in turn only includes an old version of the Xen guest-utilities and they don't seem to be so eager on updating.

RancherOS could be a winner here if the Xen guest-utilities were included from start. XenServer is a great product with many functions for free that cost a bundle of $$$ from other vendors and RancherOS would be a great Docker complement to it.

ibuildthecloud commented 8 years ago

Since we are very size conscious it would be hard in include this in the core. Any go program is typically gigantic. What we could do is try to dockerize this such that we could pull it in as a container. Having quite a bit of experience with XenServer in the past, I would love to support this platform.

ExecDevOps commented 8 years ago

Oh, I did not realize that the files on GitHub were written in go. Strange, because the files in the xs-tools.iso in the XenServer .ISO are written in bash. I wonder what Citrix is up to, the files have same name, etc but one set is written in bash-script and the other in go?!?

My original request was ofcourse for the bash-scripts on the XenServer installation .ISO. But as long as you guys make sure to include Xen support in the kernel than one can install (hack-install) the guest tools manually and thus make RancherOS become a well integrated XenServer VM.

ibuildthecloud commented 8 years ago

The shell scripts would probably be fine to include. Those are what I'm used to. The go tools were new to me also. We will investigate this.

ExecDevOps commented 8 years ago

Well, if you can fix whatever thingy that hinders Docker to start on RancherOS then I can volunteer to throw myself at implementing the bash-script version of XenServer Tools. Or at least give it a try or two. :-)

Klowner commented 8 years ago

Not sure if this is helpful or even fully functional but I whipped this up yesterday and seems to work reasonably well for me so far.

https://github.com/Klowner/rancher-xe-guest-utils

I'm largely unfamiliar with the xenstore stuff, but it's included in the container (I assume xe-daemon uses it in some manner).

ExecDevOps commented 8 years ago

Hi Klowner, thank you for your effort, it seems to be working well! I setup a VM in Xenserver of the "Other install media" type, let it boot RancherOS via iPXE and made it install to /dev/xvda.

My PXE menu entry for RancherOS looks like this, it loads files extracted from the latest RancherOS .ISO:

KERNEL http://nas.my.com/pxe/rancheros/vmlinuz rancher.password=rancher rancher.state.autoformat=[/dev/xvda] rancher.cloud_init.datasources=[url:http://nas.my.com/pxe/rancheros/cloud-config.yml]
INITRD http://nas.my.com/pxe/rancheros/initrd

The http://nas.my.com/pxe/cloud.config file contains info regarding your service as well as my SSH key, I am not sure that they are needed at this stage but it did not do any harm:

#cloud-config.yml
ssh_authorized_keys:
  - ssh-rsa AAAA...

hostname: rancheros-1

rancher:
  services_include:
     https://raw.githubusercontent.com/Klowner/rancher-xe-guest-utils/master/xe-guest-utils.yml: true

Once booted I edited /var/lib/rancher/conf/cloud-config file and made sure that my SSH key + your service is in there, i.e. the file has the same contents as above.

Thereafter I installed RancherOS to the harddrive:

sudo ros install -c /var/lib/rancher/conf/cloud-config.yml -d /dev/xvda

After reboot from the harddrive the VM showed up as having Xenserver Tools 6.6 installed and seems to be a happy member of the Xenserver genuine VM family.

Adding this VM to the Rancher management was as easy, just pasting the command for adding a host as shown in Rancher. I will play around with RancherOS and see if I can incorporate it into Xencenter's newborn support for Docker.

Many thanks!

joshwget commented 8 years ago

@Klowner Nice work! I think your implementation would be great as an official service. If you're interested, I'd be happy to help you integrate it into the rancher/os-images and rancher/os-services repos.

Klowner commented 8 years ago

@joshwget sure! That would be great.

I think I've got a couple outstanding issues as well, it'd be nice to have some assistance from someone who is more familiar with rancher internals.

TheNicholasNick commented 7 years ago

Am keen to run RancherOS as a PV guest under xenserver. (6.5sp1 as dedicated provider isn't able to supply kvm/ipmi to upgrade to 7.0)

Will stick with HVM for the moment.

Had a quick look around the xen-tools, however not really sure what needs to be supplied to a docker container for it all to work.

Suspect that rancheros default of ext4 is going to cause issues and will need a ext2 /boot partition (unless that was old info from previous versions of xenserver)

Maybe this will get a look in for > v0.8.x be cool to have rancheros fully functional on raspberry pi's after all ;)

SvenDowideit commented 7 years ago

@SoreGums that'd be very much appreciated - I'm working on porting the installation code to go for 0.8 atm, and we can work towards having it work in 0.9.

TheNicholasNick commented 7 years ago

I've been groking the xen utilities source and there is not really that much happening...

This is what I have managed to figure out so far.

Auto Install (iso > Linux > install.sh) only supports Red Hat/Cent OS, Debian/Ubuntu, SuSe, Boot2Docker out of the box. This is done by xe-linux-distribution which looks for specific files. _(in rancheros using the ubuntu console, it doesn't like what it finds in /etc/debianversion > sketchy/sid, so fails out, you can override and provide the correct details) Once that is figured out, files get put in place as per that distros "way". There is an init script for xe-linux-distribution as well that writes info about linux distro into xenstore via xe-daemon (that it also starts). xe-daemon is what we need to be working/started up. xe-daemon then uses xenstoreclient to talk to xen host and needs access to xenbus which should be located on one of these paths

  1. /proc/xen/xenbus
  2. /dev/xen/xenbus
  3. /kern/xen/xenbus

Finally there is a hotplug rule to handle adding and removing vcpu's.

Now that we know what is needed, it is just a matter of figuring out the service container contents:

  1. exposing xenbus to this container
  2. applying a patch to xe-linux-distribution
    • thus parsing /etc/os-release info correctly
  3. make sure that the golang os/signals are accessible to this container and likewise received by the rest of system-docker when sent
  4. there is a stats collector, it needs access to that info
  5. handle the hotplug rules to enable vcpu adding/removing
  6. ?? not sure how the memory balloon driver works, that might be a kernel thing that is just taken care of by the kernel??

OK so here are the sizes of the currently (v7.0.0 [1]) compiled golang files plus the init and hotplug script from the debian package and where they are located

file size (bytes)
/etc/init.d/
xe-linux-distribution 2,710
/lib/udev/rules.d/
z10_xen-vcpu-hotplug.rules 114
/usr/sbin
xe-linux-distribution 9,666
xe-daemon 1,810,392
/usr/bin/
xenstore-write 17
xenstore-rm 17
xenstore-read 17
xenstore-exists 17
xenstore 1,307,672
TOTAL 3,130,577

Rightio, mind dump finished. Not sure if I'll be able to put together a working container, will tinker and see what happens.

[1] you need to download the XenServer-7.0.0-install-cd.iso from Citrix and then you'll find guest-tools-7.0.0.iso here > XenServer-7.0.0-main.iso\Packages\xenserver-pv-tools-7.0.0-125243c.2061.noarch.rpm\xenserver-pv-tools-7.0.0-125243c.2061.noarch.cpio.\opt\xensource\packages\iso\ (can use 7-zip to drill down and extract, this path is from 7-zip on Windows)

gabrielbaez commented 7 years ago

I know this is an old threat, but did this ever get completed ?

DevKyleS commented 7 years ago

Ditto, wondering if this is still in the plans to achieve soon or not. Thanks!

gmadrigalcr commented 7 years ago

Hi,

Any news on it?

Klowner commented 7 years ago

Sorry! I've been swamped with a ton of other things, if anyone wants to dig into this more they're more than welcome to. This never got merged with any official stuff, and I haven't been doing much work with Xen lately.

pavel-hladik commented 6 years ago

Good day,

I have tried RancherOS as VM in XCP-ng hypervizor 7.4.1. OS is running well with PV drivers in HVM mode, as it use modern kernel.

We have a one of the best hypervisors back fully opensourced thanks to guys from XenOrchestra with project https://xcp-ng.org .

Do you think this project of implementation of xs-tools for RancherOS is totally dead and we should focuse on other OS?

It's a shame, cause RancherOS is great and supports KVM qemu, VMware vm-toolsd.

Pavel

Klowner commented 6 years ago

@kaitik does https://github.com/Klowner/rancher-xe-guest-utils not work for you?

pavel-hladik commented 6 years ago

I have tried your xs-utilities with XCP-ng 7.4.1 and it seems not working.

Klowner commented 6 years ago

If had a functional Xen server and I knew how to boot rancher with a cloud config I'd be willing to take a look, but as it is, I am without those things.

hrvatskibogmars commented 4 years ago

Any news regarding rancher-xe-guest-utils? Did anyone manage to get it up and running?

zombiehoffa commented 2 years ago

Wouldn't want to interrupt the 2 year cycle of asking why this isn't implemented yet, so any news?

ExecDevOps commented 2 years ago

Excellent question, I agree!

Den lör 25 juni 2022 20:08zombiehoffa @.***> skrev:

Wouldn't want to interrupt the 2 year cycle of asking why this isn't implemented yet, so any news?

— Reply to this email directly, view it on GitHub https://github.com/rancher/os/issues/813#issuecomment-1166336374, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDPPK6AVYUPD4FTWM5MXZ3VQ5DKBANCNFSM4B6TEKGQ . You are receiving this because you authored the thread.Message ID: @.***>