openflighthpc / concertim-ansible-playbook

Ansible playbook for building a Concertim appliance
Eclipse Public License 2.0
0 stars 0 forks source link

Improve experience when upgrading concertim deployment #85

Open benarmston opened 8 months ago

benarmston commented 8 months ago

When upgrading concertim et al there are two situations that could require the images to be rebuilt and the containers to be recreated.

  1. The git repository for the service, e.g., visualiser or the middleware, has been updated.
  2. The git repository for the ansible playbook, i.e., this repository, has been updated.

Currently, the playbook automatically detects if the service repository has been updated, and rebuilds the image/recreates the container appropriately. That is case (1) above is handled correctly.

The playbook does not currently, detect that the ansible playbook (this repo) has been updated, and does not automatically rebuild the image/recreate the containers. That is case (2) above is not automatically handled and requires user intervention.

The user intervention is as follows:

concertim_images=$(docker compose -f /opt/concertim/opt/docker/docker-compose.yml ps --all --format '{{.Image}}' | grep '^concertim-')
docker compose -f /opt/concertim/opt/docker/docker-compose.yml down
for i in ${concertim_images} ; do docker image rm $i ; done

With that done the playbook can be re-ran and will create new images appropriately.


For many upgrades, upgrading the playbook will go hand-in-hand with upgrading the services, however this is not necessarily the case. The most reliable way to ensure that the images / containers are rebuilt as necessary, would be to automate removing the existing containers and images. A more targetted approach that attempts to detect when this is necessary and thereby limit the down time might be desirable in the long term, but for now a simpler approach is preferable.

A new playbook should be added to this repo to remove the concertim containers and images, such that upgrading an installation is done as follows:

cd /opt/concertim/opt/ansible-playbook/ansible

ansible-playbook \
  --inventory inventory.ini \
  --extra-vars "gh_token=$GH_TOKEN" \
  --extra-vars @etc/globals.yaml \
  down.yml

ansible-playbook \
  --inventory inventory.ini \
  --extra-vars "gh_token=$GH_TOKEN" \
  --extra-vars @etc/globals.yaml \
  playbook.yml

or perhaps

cd /opt/concertim/opt/ansible-playbook/ansible

ansible-playbook \
  --inventory inventory.ini \
  --extra-vars "gh_token=$GH_TOKEN" \
  --extra-vars @etc/globals.yaml \
  upgrade.yml
LoganS56 commented 8 months ago

@benarmston this is very useful - I will make note of these instructions and be sure to send them out if needed.