lxc.init fails to bootstrap container

[cheuschober]

Happy to be proven wrong here, but from my best understanding, calls to lxc.init will always fail as long as it attempts to remove the seed marker prior to starting the container.

remove_seed_marker is set to True at: https://github.com/saltstack/salt/blob/develop/salt/modules/lxc.py#L1197

The bit that looks like it could use some reordering is: https://github.com/saltstack/salt/blob/develop/salt/modules/lxc.py#L1230-L1259

Steps to reproduce:

$ salt-call lxc.init testbuntu template=ubuntu options="{'release':'trusty', 'arch': 'amd64'}"

First failure is in attachable()

[ERROR   ] Command 'lxc-attach --clear-env -n testbuntu -- /usr/bin/env' failed with return code: 1

Versions report:

$ salt --versions-report
                  Salt: 2015.5.0
                Python: 2.7.6 (default, Mar 22 2014, 22:59:56)
                Jinja2: 2.7.2
              M2Crypto: 0.21.1
        msgpack-python: 0.3.0
          msgpack-pure: Not Installed
              pycrypto: 2.6.1
               libnacl: Not Installed
                PyYAML: 3.10
                 ioflo: Not Installed
                 PyZMQ: 14.0.1
                  RAET: Not Installed
                   ZMQ: 4.0.4
                  Mako: 0.9.1
 Debian source package: 2015.5.0+ds-1trusty1

[kiorky]

The command i used for the deletion:

salt-cloud -y -lall -p lxct -d foobar5

[kiorky]

yep, you are missing the profile on the deletion command.

[kiorky]

even without profile, i dont reproduce.

lxct:
    ----------
    lxc:
        ----------
        foobar6:
            ----------
            comment:
                foobar6 was destroyed
            result:
                True

[kiorky]

btw i think that the path support, for your use case (testing) may interrest you as the main use case is to spawn the container on a previously mounted tmpfs, thus speeding up the creation of containers and making the wipe easy on test envs.

[kiorky]

This feature was mainly asked by @jpic

[kiorky]

@jpic all the pr ive done are on the way to be merged, (those for 2015.5 are in) and #23982 is going to be merged in the next days.

[kiorky]

@cheuschober do you still have any issue ?

[cheuschober]

@kiorky I do but let me do just a little more testing to confirm it's not me -- the issue I had was partly with bootstrap. I had minions pre-seeded in my lxc templates and the bootstrap script wasn't taking the "-C" argument to just set master and other configuration options. I hadn't formally reported it if only because this could be an edge-case with the bootstrap script's autodetection or with the lxc runner's seeding options and I wanted to be sure.

[kiorky]

@cheuschober did you saw #24126 , that may be of interrest.

[cheuschober]

@kiorky Apologies for taking so long to get back to this but I finally found some time to start debugging and will be doing so all weekend. The first bug I encountered was using lxc init with a default bridge on lxc v1.0.7

In this case the config does not have lxc.network.ipv4 and it is illegal to have lxc.network.ipv4.gateway when lxc.network.ipv4 is not set (container will not start).

A (potential) suggested fix is:

843c843,844
<         True not in ['ipv4.gateway' in a for a in ret]
---
>         True not in ['ipv4.gateway' in a for a in ret] and
>         True in ['ipv4' in a for a in ret]

[cheuschober]

I think that's the last major bug I encountered on the 2015.5.X branch.

[jpic]

That patch does indeed let salt-cloud start lxc instances, but then it fails as such:

[ERROR   ] Command 'lxc-attach --clear-env  --set-var PATH=/bin:/usr/bin:/sbin:/usr/sbin:/opt/bin:/usr/local/bin:/usr/local/sbin -n test26 -- if [ -f "/sbin/67cee62104466d2448ab8543428ec4c0_dns.sh" ];then rm -f "/sbin/67cee62104466d2448ab8543428ec4c0_dns.sh";fi' failed with return code: 1
[ERROR   ] stderr: /bin/bash: -c: line 0: syntax error near unexpected token `then'
/bin/bash: -c: line 0: `lxc-attach --clear-env  --set-var PATH=/bin:/usr/bin:/sbin:/usr/sbin:/opt/bin:/usr/local/bin:/usr/local/sbin -n test26 -- if [ -f "/sbin/67cee62104466d2448ab8543428ec4c0_dns.sh" ];then rm -f "/sbin/67cee62104466d2448ab8543428ec4c0_dns.sh";fi'

Apparently the "then" statement is outside the command because of ;....

[aleufroy]

Hi,

I'm not really sure it's related but I have troubles with lxc-cloud also:

/etc/salt/cloud.providers.d/localhost.provider.conf:

localhost-lxc:
  target: padian
  driver: lxc

/srv/salt/containers.sls:

nginx_container:
  cloud.present:
    - cloud_provider: localhost-lxc
    - template: archlinux
    - running: true
    - type: veth
    - flags: up
    - link: lxcbr0
    - gateway: 10.0.3.1
    - ip: 10.0.3.2
    - master: 10.0.3.1

Then I performed a salt padian state.apply containers which results in a "Invalid return for nginx_container: ERROR: nginx_container is not attachable. {'padian': {'ret': 'ERROR: nginx_container is not attachable.', 'out': 'nested'}}".

Looking in the log of the minion I found an python traceback ending with:

File "/usr/lib/python2.7/site-packages/salt/modules/lxc.py", line 3678, in _run
  '{0} is not attachable.'.format(name)

After some code inspection, I found that the minion fails to execute the following command (return code -11):

$ lxc-attach --clear-env --set-var PATH=/bin:/usr/bin:/sbin:/usr/sbin:/opt/bin:/usr/local/bin:/usr/local/sbin  -n nginx_container -- df /

This command succeed when I Iaunch it manually, returning:

Filesystem               1K-blocks     Used Available Use% Mounted on
/dev/mapper/storage-root 482316904 29053184 428740356   7% /

But It fails when I try:

$ salt padian cmd.run 'lxc-attach --clear-env --set-var PATH=/bin:/usr/bin:/sbin:/usr/sbin:/opt/bin:/usr/local/bin:/usr/local/sbin  -n nginx_container -- df /'
padian:

I've to specify use_vt=true:

salt padian cmd.run 'lxc-attach --clear-env --set-var PATH=/bin:/usr/bin:/sbin:/usr/sbin:/opt/bin:/usr/local/bin:/usr/local/sbin  -n nginx_container -- df /' use_vt=true
padian:
    Filesystem                      1K-blocks     Used Available Use% Mounted on
    /var/lib/lxc/jessie-base/rootfs 482316904 29611404 428182136   7% /

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.