search

canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.34k stars 930 forks source link

17.10 artful and cloud-init #3954

Closed jfgibbins closed 6 years ago

jfgibbins commented 6 years ago

Attempted to spin up a container using artful from ubuntu-daily in order to do some testing in regards to netplan/cloud-init. I noticed that there were no cloud-init files at the usual /var/lib/cloud/seed/nocloud-net, nor was there even a /var/lib/cloud directory. Where did they go?

stgraber commented 6 years ago

Seems fine here.

stgraber commented 6 years ago 
stgraber@castiana:~$ lxc launch ubuntu-daily:17.10 artful
Creating artful
Starting artful                                
stgraber@castiana:~$ lxc exec artful bash
root@artful:~# ps fauxww
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root       122  0.0  0.0  21912  2492 ?        Ss   17:54   0:00 bash
root       229  0.0  0.0  42372  2228 ?        R+   17:54   0:00  \_ ps fauxww
root         1  0.0  0.0 154432  5796 ?        Ss   17:54   0:00 /sbin/init
root        52  0.0  0.0  41752  1908 ?        Ss   17:54   0:00 /lib/systemd/systemd-udevd
root        57  0.0  0.0  65120  6252 ?        Ss   17:54   0:00 /lib/systemd/systemd-journald
systemd+   118  0.0  0.0  74476  3368 ?        Ss   17:54   0:00 /lib/systemd/systemd-networkd
root       162  0.0  0.0  31672  1864 ?        Ss   17:54   0:00 /usr/sbin/cron -f
root       163  0.0  0.0  57016  3412 ?        Ss   17:54   0:00 /lib/systemd/systemd-logind
syslog     164  0.0  0.0 187036  2408 ?        Ssl  17:54   0:00 /usr/sbin/rsyslogd -n
root       166  0.0  0.0 287800  4284 ?        Ssl  17:54   0:00 /usr/lib/accountsservice/accounts-daemon
message+   167  0.0  0.0  47452  2428 ?        Ss   17:54   0:00 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
systemd+   170  0.0  0.0  65584  3860 ?        Ss   17:54   0:00 /lib/systemd/systemd-resolved
root       174  0.0  0.0 689556 12412 ?        Ssl  17:54   0:00 /usr/lib/snapd/snapd
daemon     175  0.0  0.0  28256  1592 ?        Ss   17:54   0:00 /usr/sbin/atd -f
root       176  0.0  0.0 288652  4308 ?        Ssl  17:54   0:00 /usr/lib/policykit-1/polkitd --no-debug
root       202  0.0  0.0  72504  2428 ?        S    17:54   0:00 (agetty)
root       205  0.0  0.0  72148  3660 ?        Ss   17:54   0:00 /usr/sbin/sshd -D
root       219  0.0  0.2 103772 33920 ?        Rs   17:54   0:00 /usr/bin/python3 /usr/bin/cloud-init modules --mode=final
root       224  0.0  0.0   4608   608 ?        S    17:54   0:00  \_ /bin/sh -c tee -a /var/log/cloud-init-output.log
root       225  0.0  0.0   7876   636 ?        S    17:54   0:00      \_ tee -a /var/log/cloud-init-output.log
root@artful:~# ls -lh /var/lib/cloud/
data/      handlers/  instance/  instances/ scripts/   seed/      sem/       
root@artful:~# ls -lh /var/lib/cloud/seed/
total 8.5K
drwxr-xr-x 2 root root 6 Oct 18 17:54 nocloud-net
root@artful:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 17.10
Release:    17.10
Codename:   artful
root@artful:~#
stgraber commented 6 years ago

You used ubuntu-daily: and not images: right?

jfgibbins commented 6 years ago

correct lxc init ubuntu-daily:artful test -p default -p test lxc start test copy/pasted from cli history cloud-init was there, but not the directories until I ran cloud-init init now, went back, did it as a launch per your example, and it was there like yours, copied my history doing init then start, and it's missing again. Did I miss something there? Hasn't launch always just been init/start combined? Always seemed to be that way for me before.

stgraber commented 6 years ago 
stgraber@castiana:~$ lxc init ubuntu-daily:17.10 artful
Creating artful
stgraber@castiana:~$ lxc start artful
stgraber@castiana:~$ lxc exec artful bash
root@artful:~# ls -lh /var/lib/cloud/
data/      handlers/  instance/  instances/ scripts/   seed/      sem/       
root@artful:~# ls -lh /var/lib/cloud/seed/
total 8.5K
drwxr-xr-x 2 root root 6 Oct 18 18:04 nocloud-net
root@artful:~# ls -lh /var/lib/cloud/seed/nocloud-net/
total 6.0K
-rw-r--r-- 1 root root  58 Oct 18 18:04 meta-data
-rw-r--r-- 1 root root 121 Oct 18 18:04 network-config
-rw-r--r-- 1 root root  18 Oct 18 18:04 user-data
-rw-r--r-- 1 root root  63 Oct 18 18:04 vendor-data
root@artful:~#
jfgibbins commented 6 years ago

Alright, this is very weird. Noticed the one thing I had different than you, was I was using ubuntu-daily:artful and you had ubuntu-daily:17.10. And we had different results. I switched to 17.10 as the name like yours, copy/paste, and of course worked. Then tested again with init/start, using 17.10, it worked. Went back to ubuntu-daily:artful, launch and init/start, both now working. What the hell changed to the image by switching to 17.10 instead of artful? Also, did a lxc image list ubuntu-daily | grep artful. There was nothing, so how the hell was it creating a container in the first place? Did it actually ignore the image server name, and use the previous artful that had been downloaded early today from images:? Ubuntu-daily has 17.10, but not artful. images: has artful, but not 17.10.

stgraber commented 6 years ago

ubuntu-daily has the same image under various names, you can get the list with "lxc image info ubuntu-daily:17.10"

stgraber commented 6 years ago

17.10, 17.10/amd64, a, a/amd64, artful, artful/amd64, devel, devel/amd64 all point to the same thing in theory

jfgibbins commented 6 years ago

Definately agreed. Not sure why that happened. It really seems as if my attempts prior to using 17.10 in place of artful, was really not wanting to give up the cached image from images. Maybe I'll clear all the images from the server later today, and try it all over again to see if I can replicate it, if nothing else for my own sanity. I'm still annoyed today from the whole 17.10 switching to netplan. I'm sure there's a reason for it, but til find it, it's just new syntax to remembers. And changing muscle memory for nano /etc/netw<tab>/in<tab>.<tab><tab><enter>. Yes, I love autocomplete. Though cisco ios still has it beat, as I dont't even have to tab out the command for it to work.

stgraber commented 6 years ago

Closing this as there doesn't appear to be an actual LXD issue. You may have had an older image that got returned from cache which had a busted cloud-init, that or cloud-init is randomly breaking somehow.

davidfavor commented 6 years ago

This issue should likely be reopened, as something is broken in the artful template.

I just created an artful container + many /etc/network files are missing + the /etc/network directory is never touched (using inotifywait at host level), when container spins up, so networking never comes online.

lxc init ubuntu:artful/amd64 net11-ubuntu-artful

Produces an /etc/network which is barely populated.

net11 # find /var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/interfaces
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-post-down.d
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-post-down.d/vlan
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-up.d
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-up.d/ethtool
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-up.d/openssh-server
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-up.d/ip
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-pre-up.d
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-pre-up.d/vlan
/var/lib/lxd/containers/net11-ubuntu-artful/rootfs/etc/network/if-pre-up.d/ethtool

My host config...

net11 # lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 17.04
Release:    17.04
Codename:   zesty

net11 # lxd --version
2.18
davidfavor commented 6 years ago

I've tried running cloud-init init as stated above. /etc/network still not referenced in container start sequence.

davidfavor commented 6 years ago

Deleted all cached images + regenerated container + same problem persists.

davidfavor commented 6 years ago

Arg... This does appear to be an artful-ism, because this works...

lxc init ubuntu-daily: net11-ubuntu-daily

/etc/network is populated correctly when using ubuntu-daily as the image.

Although, checking inside the container, it appears ubuntu-daily is munged also, as ubuntu-daily spins up a xenial container...

lxc exec net11-ubuntu-daily bash

root@net11-ubuntu-daily:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:    16.04
Codename:   xenial
jfgibbins commented 6 years ago

@davidfavor FYI, in artful, /etc/network and the packages with ifconfig, etc, have been deprecated in favor of netplan and the "ip xxx" commands. google or yahoo search for "artful netplan" will bring up quite a bit of info. https://wiki.ubuntu.com/MigratingToNetplan. Also, cloud-init does work with it. In fact for my network-config files it handled them pretty well. Kind of a pain, as currently, netplan is not quite as capable, but, it is a non-lts, hopefully they'll have it finished by the time 18.04 LTS comes out. Sort of like with ZFS when it came out in 15.10.

davidfavor commented 6 years ago

Thanks.

If you have any idea of how to bring up networking in an Artful container, please post instructions.

Haven't figured it out yet.

Ah... I see... Looks like some .yml file under /etc/netplan is required to describe public IPs.

If you have an example of how you're doing this, post your /etc/netplan files.

Thanks for the tip. I was pulling my hair out.

jfgibbins commented 6 years ago

@davidfavor if you're using version 1 or version 2 for networking, http://cloudinit.readthedocs.io/en/latest/topics/network-config.html#network-configuration-sources, then it should work just fine. If you're using ENI, I"m not quite sure, as I haven't tested with it, but I don't believe it does. The format of eni is strictly written for ifupdown and doubtful it has a conversion to version 1/2 for netplan to consume. Cloud-init should create cat /etc/netplan/50-cloud-init.yaml

davidfavor commented 6 years ago

I'm going to just open another issue about this, as likely this will trip up many people.

Take a look at the new issue + if post any examples of /etc/netplan .yaml files you have.

Thanks.

jfgibbins commented 6 years ago

If you're using cloud-init, you won't write any /etc/netplan files. Cloud-init does it for you automatically. You just have to supply the network-config file as version 1 or version 2 format. Cloud-init makes the file just like it did for /etc/network. As for manually writing netplan, there's lots on the internet since it's been in ubuntu since 16.10.

weycrest commented 6 years ago

I was googling and came across this discussion. Having a similar issue with 17.10 on a scaleway VPS which I run LXD on. I can create LXD containers at home on my Ubuntu Mate 17.10 desktop, but try as I might I can't on the scaleway server. 16.04 and 17.04 images are fine. I can even push these images. What happen is that a very scarcely populated container gets created with a few processes running (you can get a bash shell) but that is all. I've tried every combination of creating or launching a VPS and trying different variations (amd64 etc) to no avail. Pushing a working 17.10 to the scaleway server doesn't work either. You just get a sparce VPS which can't pull an IP address, and has to be forcibly shutdown. Not a big issue for me as I can continue working with 16.04, just noting for the developers of this excellent project. The scaleway instance is running 16.04 by the way with zfs via a loopback device