Upgrade to Ubuntu 22.04, Python 3.10

[rajadain]

Overview

Upgrades the development and runtime VMs to Ubuntu 22.04, and the environment to Python 3.10. This is done in preparation for #3628 which needs Python 3.10 for pystac-client.

The big consequences of this change are:

• The nginx role is now inlined model-my-watershed.nginx instead of azavea.nginx because with new Ubuntu we don't need the custom PPA that the azavea.nginx role was installing, and inlining it here was easier than updating it there.
• We now use Chromium instead of Firefox for JavaScript tests, because installing Firefox was difficult, and because Chromium is a better representative of our userbase
• RWD is switched to run docker as a system service rather than through the outdated docker-compose Python wrapper
  • Notably, we upgraded docker to v26 which is not compatible with the old RWD Docker v1 image. This causes RWD to fail. We have https://github.com/WikiWatershed/rapid-watershed-delineation/issues/83 to fix this as a follow up.

Closes #3629

Demo

Here's this branch deployed to staging: https://staging.modelmywatershed.org/

$ xh staging.modelmywatershed.org/version.txt

HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 100
Content-Type: text/plain
Date: Wed, 03 Jul 2024 18:55:05 GMT
Etag: W/"66858e35-59"
Last-Modified: Wed, 03 Jul 2024 17:45:25 GMT
Server: nginx
Vary: Accept-Encoding

origin/tt/3629/upgrade-ubuntu-python 1.35.0-26-g16a4920ca31cd2a000cac1e7970ec8de4896c706

Notes

This required the updating of GWLF-E to work with Python 3.10 https://github.com/WikiWatershed/gwlf-e/pull/95 and subsequent release to PyPI https://pypi.org/project/gwlf-e/3.2.0/

This also required significant changes / upgrades to the CivicCI01 Jenkins server. Model My Watershed is the only active project on that server, which made making changes a little safer. All the CI jobs are now passing: http://civicci01.phl.corp.element84.com/view/mmw/

Testing Instructions

[!NOTE] Delineating watersheds will not work on staging right now. This is a known issue https://github.com/WikiWatershed/rapid-watershed-delineation/issues/83

• Go to https://staging.modelmywatershed.org/version.txt
  • [x] Ensure you see a reference to this branch, not develop
• Go to https://staging.modelmywatershed.org/
  • [x] Ensure selecting a shape to draw works
  • [x] Ensure analyzing a shape works
  • [x] Ensure Watershed Multi Year Model works
  • [ ] Ensure Site Storm Model works

[!WARNING] These instructions will take multiple hours to run and will fully engage your computer. Reviewers are advised to trigger the commands during off hours and check back later to see progress.

• Check out this branch, and destroy and recreate the VMs, and install necessary data:

  vagrant destroy -f && vagrant up &&
  vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -bc' &&
  vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -dmpq' &&
  vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -sS' &&
  vagrant reload

  • [x] Ensure it finishes successfully
• Go to http://localhost:8000/
  • [x] Ensure MMW behaves as normal

[rajadain]

The geoprocessing service should be fixed in 45e4f4d. I haven't seen the other error before, running that command now to see if I encounter it.

[rajadain]

Destroy and recreated the app VM, now seeing that error too. Likely an issue with gwlf-e packaging. Will look into it.

[rajadain]

The numpy issues should be fixed in 4c1fe4f and be5b2c0, as already reviewed in https://github.com/WikiWatershed/gwlf-e/pull/101#pullrequestreview-2164557941.

Since you've already provisioned the VM for the review above, try running the failing command above again:

vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -dmpq'

If it succeeds, then we can probably approve this PR for now, especially because this version is also deployed to staging and is testing well there: https://staging.modelmywatershed.org/.

You'll have to do reinstall the other datasets for the future, but that is a time taking operation and can be deferred till after this review:

vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -bc' &&
vagrant ssh app -c 'cd /vagrant && ./scripts/aws/setupdb.sh -sS'

[rajadain]

Thanks for reviewing! Squashed fixups, will merge once green.