Update to 16?

[dereuromark]

Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-24-generic x86_64)

...

New release '16.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Any way to have 16 by default already?

[bravo-kernel]

That would require generating a new box using https://github.com/alt3/cakebox-builder. I have no short term plans so a manual upgrade would be the quickest route atm.

[dereuromark]

For security reasons it soon becomes vital that the box ships with current defaults and mainly also php7.0+ (better 7.1) by default.

[josegonzalez]

This is used for local development. What security reasons are there around upgrading this now?

[bravo-kernel]

In all honesty this PR could hypothetically introduce security issues but IMHO those are far-fetched as the intended usage is for (really local) LAN only and holds no ground as a must-upgrade-now argument. Besides that I see:

• no short-term upgrade of the box-version due to the complex Chef kitchen build which will require serious time (feel free to try it yourself)
• no need to implement PHP7, one is free to install it themselves on the box (using customization script in YAML)

Lastly, if PHP gets touched it will no longer be single-version oriented but instead implement one of the many solutions out there supporting multiple PHP versions (the single design error we made when thinking up the box).

[bravo-kernel]

One more thing to hopefully prevent this from sounding to negative/unwilling. While adding the backup functionality I thought about adding a (new) bash upgrade script instead of creating a new box. IMO this would be the desired route since it would mean existing cakebox could keep using their existing boxes.

E.g. executed after some boolean switch in the yaml (e.g. autoUpdate). This would then also be usable for upgrading/replacing PHP, updating sql, security patches, etc.

[bravo-kernel]

I just managed to do an in-box upgrade so this feature is surely coming now, PHP 7.1 included.

I will need some time to integrate this with cakebox provisioning but these are the steps if you want to do the upgrade manually;

## Remove this directory as it will prevent do-release-upgrade building the new kernel image
sudo rm /etc/udev/rules.d/70-persistent-net.rules/ -rf

## Run dist-upgrade to upgrade installed packages and build new kernel
## image as preparation for major version upgrade (--confold to prefer
## keeping existing confs to not break e.g. IP configuration)
sudo DEBIAN_FRONTEND='noninteractive' apt-get -y -o Dpkg::Options::='--force-confdef' -o Dpkg::Options::='--force-confold' dist-upgrade

## ============================================
## FYI software has already been upgraded here:
## 
## lsb_release -a   => 16.04.2 LTS
## php -v       => php 7.1.3
## hhvm --version   => hhvm 3.18.1
## nginx -v     => nginx 1.11.9
## ============================================

## Remove no longer required packages and clean up apt
sudo apt-get autoremove --assume-yes
sudo apt-get clean --assume-yes
sudo apt-get autoclean --assume-yes

## Make sure release-upgrade (and java) don't block on required user-input
echo 'DPkg::options { "--force-confdef"; "--force-confmiss"; }' | sudo tee /etc/apt/apt.conf.d/local

## Upgrade to 16.04 LTS (not using DistUpgradeViewNonInteractive because of lacking console feedback)
sudo sh -c 'echo "y\ny\ny\ny\n" | DEBIAN_FRONTEND=noninteractive /usr/bin/do-release-upgrade'

## Cakebox specific cleanup:
sudo rm /etc/nginx/sites-available/default.dpkg-dist
sudo rm /etc/update-motd.d/10-help-text
sudo rm /etc/apt/apt.conf.d/50unattended-upgrades.ucf-dist

## ==================
## === snapshot =====
## ==================

## =========================================================
## Install php7.1-fpm and re-install now missing 7.1 modules
## =========================================================
sudo add-apt-repository ppa:ondrej/php --yes
sudo apt-get update
sudo apt-get autoremove --assume-yes
sudo apt-get clean --assume-yes
sudo apt-get autoclean --assume-yes

sudo apt-get install php7.1-fpm --assume-yes

sudo apt-get install php7.1-apc --assume-yes
sudo apt-get install php7.1-bcmath --assume-yes
sudo apt-get install php7.1-bz2 --assume-yes
sudo apt-get install php7.1-curl --assume-yes
sudo apt-get install php7.1-dba --assume-yes
sudo apt-get install php7.1-dom --assume-yes
sudo apt-get install php7.1-gd --assume-yes
sudo apt-get install php7.1-gearman --assume-yes
sudo apt-get install php7.1-geoip --assume-yes
sudo apt-get install php7.1-gmp --assume-yes
sudo apt-get install php7.1-imagick --assume-yes
sudo apt-get install php7.1-imap --assume-yes
sudo apt-get install php7.1-intl --assume-yes
sudo apt-get install php7.1-json --assume-yes
sudo apt-get install php7.1-mbstring --assume-yes
sudo apt-get install php7.1-mcrypt --assume-yes
sudo apt-get install php7.1-memcache --assume-yes
sudo apt-get install php7.1-memcached --assume-yes
sudo apt-get install php7.1-mysql --assume-yes
sudo apt-get install php7.1-mysqli --assume-yes
sudo apt-get install php7.1-readline --assume-yes
sudo apt-get install php7.1-redis --assume-yes
sudo apt-get install php7.1-soap --assume-yes
sudo apt-get install php7.1-sqlite3 --assume-yes
sudo apt-get install php7.1-xdebug --assume-yes
sudo apt-get install php7.1-xmlwriter --assume-yes
sudo apt-get install php7.1-zip --assume-yes
##sudo apt-get install php7.1-zlib --assume-yes

## Replace php5-fpm in all existing nginx vhosts and cakebox vhost-command templates
sudo find /etc/nginx/sites-available/ -type f -exec sed -i 's/php5-fpm/php\/php7.1-fpm/g' {} +
sudo find /cakebox/console/src/Template/bake/ -type f -exec sed -i 's/php5-fpm/php\/php7.1-fpm/g' {} +

## Add launchpad ppa to source.list before installing nodejs 7
cd /tmp
curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -
sudo apt-get install -y nodejs

## Re-install java (1.8)
sudo add-apt-repository ppa:webupd8team/java --yes
sudo apt-get update
echo "oracle-java8-installer shared/accepted-oracle-license-v1-1 select true" | sudo debconf-set-selections
sudo apt-get install oracle-java8-installer --assume-yes

## Remove temporary workaround to prevent user-input blocking
sudo rm /etc/apt/apt.conf.d/local

## Restart services to make sure cakebox dashboard keeps functioning
sudo service php7.1-fpm restart
sudo service nginx restart

## All done, new kernel becomes active after vm after vm reboot
## This step is optional and done running ` `vagrant reload` on local machine.
## After reboot, the following command should show kernel 4.4.0-66
uname -r

[bravo-kernel]

TODO:

• [x] php7.1-fpm
• [x] php7.1 modules
• [x] node 7.7.3, npm 4.1.2 (legacy no longer required due to ppa)
• [x] update nginx vhost templates
• [x] automatically replace php5-fpm with php7.1-fpm in existing nginx vhosts
• [x] fix java user-input breaking unattended

Skipped

• [X] percona, upgrading breaks all cakebox-specific configs, excluded from the upgrade for now

[bravo-kernel]

Input needed: any php modules you would like to see added to this current cakebox-installed-list:

sudo apt-get install php7.1-apc --assume-yes
sudo apt-get install php7.1-bcmath --assume-yes
sudo apt-get install php7.1-bz2 --assume-yes
sudo apt-get install php7.1-curl --assume-yes
sudo apt-get install php7.1-dba --assume-yes
sudo apt-get install php7.1-dom --assume-yes
sudo apt-get install php7.1-gd --assume-yes
sudo apt-get install php7.1-gearman --assume-yes
sudo apt-get install php7.1-geoip --assume-yes
sudo apt-get install php7.1-gmp --assume-yes
sudo apt-get install php7.1-imagick --assume-yes
sudo apt-get install php7.1-imap --assume-yes
sudo apt-get install php7.1-intl --assume-yes
sudo apt-get install php7.1-json --assume-yes
sudo apt-get install php7.1-mbstring --assume-yes
sudo apt-get install php7.1-mcrypt --assume-yes
sudo apt-get install php7.1-memcache --assume-yes
sudo apt-get install php7.1-memcached --assume-yes
sudo apt-get install php7.1-mysql --assume-yes
sudo apt-get install php7.1-mysqli --assume-yes
sudo apt-get install php7.1-readline --assume-yes
sudo apt-get install php7.1-redis --assume-yes
sudo apt-get install php7.1-soap --assume-yes
sudo apt-get install php7.1-sqlite3 --assume-yes
sudo apt-get install php7.1-xdebug --assume-yes
sudo apt-get install php7.1-xmlwriter --assume-yes
sudo apt-get install php7.1-zip --assume-yes
sudo apt-get install php7.1-zlib --assume-yes

[dereuromark]

Looks pretty good What I always have to install (for all the frontend things):

• npm (for bower or webpack)
• nodejs-legacy (maybe not necessary with a newer release)

[bravo-kernel]

Right, I might just pop node in there as well (if it's not too complicated).

[bravo-kernel]

node's in there too, see the todo-list

[bravo-kernel]

Assistance appreciated as I seem to be a bit in a bit of a race-condition pickle here after upgrading to php 7.1.

Seems CakeboxExecute.php still makes heavy use of conflicting Cake\Utility\String breaking vagrant provisioning;

==> default: PHP Fatal error:  Cannot use Cake\Utility\String as String because String' is a special class name in /cakebox/console/src/Lib/CakeboxExecute.php on line 11

[dereuromark]

Those should directly be replacable with Text:: calls I would say.

[bravo-kernel]

Let me see if updating the console prior to upgrading solves things, thanks

[bravo-kernel]

I just upgraded my production vm using the merged upgrade script.

Chosen approach

Due to the complexity of the upgrade I chose to :

• not integrate it with the vagrant provisioning (giving user a choice to stay on old/PHP5.6)
• use the login message to detect non-16.04 boxes
• warn users of outdated boxes
• advize them to run /cakebox/bash/ubuntu-16.sh to perform the in-box upgrade

Upgrade instructions:

1. on your local machne:
   • create a vagrant snapshot of the current box by running vagrant snapshot push
   • update the cakebox repo by running git pull
   • install PHP7 compatible cakebox-console by running vagrant reload --provision
2. log in to the box:
   • start the in-box upgrade by running /cakebox/bash/ubuntu-16.sh
   • after completion log-out
3. on your local machine:
   • reboot the box to complete the upgrade-process by running vagrant reload

Post-upgrade information

All information regarding the upgrade can be found in:

• /var/log/apt/term.log: all apt removals, installations and cleanups
• /var/log/dist-upgrade/*: all logging related to the 16.04 version upgrade

Worst-case

If things go wrong simply restore the pre-upgrade state of your box by running `vagrant snapshot pop on your local machine.

[bravo-kernel]

Feel free to test. BTW I updated the cakebox-console repo to CakePHP 3.4 so there might be some bugs in the dashboard and application. Fixes would be welcome. E.g.

[bravo-kernel]

Confirmation that the daily backups are functioning as expected would be welcome too: #76