Installer dies if a current apt-operation is in progress

[cmeisinger]

Description of Issue/Question

When using cloud init/user_data in aws, the current official Ubuntu AMI immediately updates itself on boot. This forks to the background while the system executes other cloud-init processes.

When you use the bootstrap script passed via user_data, the salt-bootstrap script will blow up if it can't immediately apt-get the necessary components, resulting in failure of the deploy.

Setup

Nothing too fancy. Pass this as user data in AWS/EC2 and watch it fail.

#!/bin/bash
echo "host.domain.lol" > /etc/hostname
hostname host.domain.lol
wget -O install_salt.sh https://bootstrap.saltstack.com
sudo sh install_salt.sh -A salt-master.server.com stable
salt-call --local grains.setval roles webserver
salt-call --local grains.setval environment production

Steps to Reproduce Issue

Attached script when passed as userdata will suffice.

Versions and Systems

Salt Version:
           Salt: 2016.11.3

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 19 2016, 06:48:10)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
        machine: x86_64
        release: 4.4.0-57-generic
         system: Linux
        version: Ubuntu 16.04 xenial

Expected Action

The bootstrapper should wait until it can grab a lock against the apt-get process and then proceed.

Workaround

Pass in the following in your user_data script...

while ! grep "Cloud-init .* finished" /var/log/cloud-init.log; do
    echo "Waiting..."
    sleep 5
done

[rallytime]

Thanks for filing this @cmeisinger. We could add some checks in the bootstrap script to look for these kinds of situations. Thank you for posting the work-around as well!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.