Updated Xenial Support PR

[vkurup]

This supercedes #138. I didn't want to hijack that one because I think some projects may be using it directly. I did, however, base my work off that PR so hopefully it will be easy for those projects to migrate once this is merged. Please review Victor's initial PR description in #138, and then some of the following comments which explain most of the changes that I made.

I have tested this branch with multiple deploys (over and over again), including the following scenarios:

• Fresh 16.04 install on a t2.small instance
• Fresh 16.04 install on a m4.large instance (which doesn't have an eth0 interface, but an ens3 one instead)
• Fresh 14.04 install
• Upgrade of a 14.04, margarita 1.7.2, salt 2015.5.5 install

Issues addressed:

• Salt 2016.3.2 doesn't work with Ubuntu 14.04. Version 2016.3.4 seems to work with both 14.04 and 16.04, so I'll be specifying that in the accompanying d-p-t PR
• Some (but not all) 16.04 installations come with network interfaces that are not named eth0, eth1, etc. I still haven't figured out the rhyme or reason to when this happens, but I know that m4.large Ubuntu instances have network interfaces named 'ens3', where the '3' can be variable. Thanks to help from @jbradberry , I changed our code to have each minion store its default_route into the salt mine, along with its list of interfaces. From those 2 values, we can get the primary ip for each minion.
• I preserved the code that allows you to specify primary_iface in the pillar. Using a pillar value isn't the best way to do this, though because each individual machine could have a different iface name, so having only 1 pillar variable doesn't really make sense. I could remove the primary_iface code entirely, but wasn't sure if it was being used by current projects.
• As mentioned by @vrocha, current versions of salt don't work with pip 8.1.1, but older versions of salt don't work with pip>8.1.1, so I added a salt-version check to decide which version of pip to use.
• @vrocha also mentioned that 2 deploys were necessary to get everything to work. This is because 16.04 now uses systemd instead of upstart, and systemd needs a 'daemon-reload' anytime an init script changes. Salt is supposed to do this automatically for you, but it doesn't work in the versions we have. So, I added a onchanges requisite to watch the init file and call daemon-reload when necessary. I added a conditional around it so that this is a no-op if we're not using systemd.
• Our supervisor init script doesn't have a command to do 'service supervisor status' and salt 2016.3.4 doesn't seem to like that. Old versions of salt didn't seem to care, but, for that reason, I upgraded the init script to one provided in Supervisor's github repo: https://github.com/Supervisor/initscripts/blob/master/ubuntu

For the upgrade of 14.04 or 12.04 projects, and for any projects currently using #138 directly, there will need to be minor changes made on the django-project-template side:

• the install_salt.sh script needs to be udpated to the latest version.
• In the fabfile, SALT_VERSION will need to be 2016.3.4
• In the fabfile, this change will need to be made to add the default_route to the mine for each minion:

  
       'mine_functions': {

• 'network.interfaces': []
• 'network.interfaces': [],
• 'network.default_route': {
• 'family': 'inet',
• }, },

Once those changes are made, you'll need to run these commands:

fab staging setup_master
fab staging setup_minion:<INSERT CURRENT ROLES HERE> -H <MINION_IP>
# repeat the setup_minion call for each minion.
fab staging deploy

Link to the accompanying d-p-t PR: https://github.com/caktus/django-project-template/pull/280

Because projects will break if they don't do the steps above, I'd recommend bumping margarita's major version, once we decide that this is the correct approach.

[vkurup]

Thanks @dpoirier. This is updated now.

[dpoirier]

Just curious, what did having fail2ban monitor the auth and mail logs have to do with getting this working on Xenial?

[vkurup]

Hmm, good question. Those were in place when I started reviewing the PR and since it worked, I focused on getting the other pieces to work. :) @vrocha wrote "Some fail2ban specific nuances that sort of worked out"

[vkurup]

I refactored the IP extraction a bit. Could you take a look and see what you think? I haven't run any real deployment tests with this yet, so I'd like to do that after I get it looking correct.

[dpoirier]

@vkurup Looks good enough. Still kind of obscure, but it's salt. (fingerscrossed) that it still works.

[vkurup]

Reconfirmed that it still works in all 3 cases:

• Ubuntu 16.04 fresh install (t2.small)
• Ubuntu 16.04 fresh install (m4.large) with dynamically named ifaces
• upgrade of Ubuntu 14.04 project using salt 2015.5.5 and margarita 1.7.2

[dpoirier]

:mans_shoe: