search

openfoodfoundation / ofn-install

Ansible scripts for provisioning and deploying Open Food Network
54 stars 112 forks source link

Add NZ to global deployment pool #775

Closed lin-d-hop closed 2 years ago

lin-d-hop commented 2 years ago

Description

NZ would like to be part of the global feature pool. They are contributing for this work, so we can prioritise it under the Funded Features Pipe.

They are on an old version of OFN so they might be a little faff.

Would be good to know what information we need from them, so that we can gather that in advance of this work being prioritised. @Matt-Yorkley or @andrewpbrett Can you help with a list of what we need from them?

Acceptance Criteria & Tests

  1. NZ are on the latest OFN version
  2. NZ are managed within the global deployment pool
Matt-Yorkley commented 2 years ago

If they've kept up with releases and they're on a recent version, it's really easy. If they're really far behind it's a bit more tricky...

Matt-Yorkley commented 2 years ago

Oooof. They're way behind...

Also I can't even SSH to the server to have a look, so someone on their end will need to provision my key and/or adjust any firewall stuff they might have (if they're on AWS they'll probably have annoyingly restrictive firewall rules by default).

Matt-Yorkley commented 2 years ago

Also worth noting: I strongly recommend they don't try to catch up quickly by just deploying the latest version. It'll need to be handled carefully and there's various pitfalls that won't be obvious.

lin-d-hop commented 2 years ago

Cool, will navigate to make these links. Will keep you posted. Thanks!

mkllnk commented 2 years ago

The current server is based on release v3.4.5 from 18 Dec 2020. It has some custom patches which are not needed any more if we configure ofn-install correctly.

WIP list of release steps

I tested applying all database migrations on each release and found some anomalies:

mkllnk commented 2 years ago

I wrote a little script to migrate the database:

#!/bin/sh
#
# Migrate database from older code base.
#
# Our current code can't run all old migrations because some of them depend
# on application code. So you need to run the migrations with the code at the
# time. Here is one example of listing releases from a certain version,
# excluding a breaking newer version and filtering a bad release:
#
#   for v in `git tag --list 'v*' --sort v:refname --contains v3.5.0 --no-contains v4.2.0 | grep -v - | grep -v v3.5.7`; do
#     ./script/migrate-old-code.sh "$v" || break
#   done

set -ex

version="$1"

# Some releases introduced a differing schema.rb file but we need the
# unmodified version to checkout the next release. Stashing is safer than just
# checking out the original schema.
git stash
git checkout "$version"

# Use correct production database:
rm config/database.yml
ln -s "$HOME/apps/openfoodnetwork/shared/config/database.yml" config/database.yml

# script/rbenv-install.sh from master
# Older releases don't contain this script.
if rbenv install --skip-existing; then
  echo "Ruby is installed."
else
  echo "Upgrading rbenv's ruby-build:"
  git -C "$(rbenv root)"/plugins/ruby-build pull

  rbenv install
fi

./script/install-bundler
bundle >> tmp/bundle.log
GOOD_MIGRATIONS=skip bundle exec rake db:migrate

My plan is to provision a new server with the most up-to-date code version, import the old database and then apply all migrations by going through all old releases.

mkllnk commented 2 years ago

Steps for the switchover:

# New server:
ansible-playbook -l nz-prod playbooks/maintenance_mode.yml

# Old server:
ansible-playbook -l nz-prod playbooks/maintenance_mode.yml -e "ansible_host=103.18.56.183"

# Log into new server:
ssh -A openfoodnetwork@eco.openfoodnetwork.org.nz

In that SSH session:

cd ~/apps/openfoodnetwork/current
sudo systemctl stop sidekiq.service && sudo systemctl stop puma.service
ssh prod.openfoodnetwork.org.nz "sudo systemctl stop sidekiq.service && sudo systemctl stop unicorn_openfoodnetwork.service"
scp -r prod.openfoodnetwork.org.nz:apps/openfoodnetwork/current/public/images/* public/images/
scp -r prod.openfoodnetwork.org.nz:apps/openfoodnetwork/current/public/spree/* public/spree/
scp -r prod.openfoodnetwork.org.nz:apps/openfoodnetwork/current/public/system/* public/system/
ssh prod.openfoodnetwork.org.nz "pg_dump -h localhost -U ofn_user openfoodnetwork | gzip" > tmp/original.sql.gz
DISABLE_DATABASE_ENVIRONMENT_CHECK=1 I_AM_SURE=1 bundle exec rails db:drop db:create
zcat tmp/original.sql.gz | psql -h localhost -U ofn_user openfoodnetwork
for v in `git tag --list 'v*' --sort v:refname --contains v3.5.0 --no-contains v4.2.0 | grep -v - | grep -v v3.5.7`; do ./script/migrate-old-code.sh "$v" || break; done
# exit

Enable the new server:

ansible-playbook -l nz-prod playbooks/deploy.yml -e "git_version=v4.1.34"
ansible-playbook -l nz-prod playbooks/maintenance_mode.yml -e "disable_maintenance=true"
mkllnk commented 2 years ago

There's some outstanding work:

drummer83 commented 2 years ago
  • Reset database as postgres user: dropdb openfoodnetwork && createdb openfoodnetworkk

@mkllnk I assume the openfoodnetworkk should only have one 'k'? Just to avoid any copy & paste mistakes.

mkllnk commented 2 years ago

Yes, I knew I should have copied it instead of typing.

My keyboard has been really annoying lately, typing things twice. It may actually be the operating system, not sure. I will upgrade soon and see.

mkllnk commented 2 years ago

The main switch is done. Some more tasks: