Upgrade ocp13 (registry) to Ubuntu 24.04 LTS

[jpmckinney]

https://ubuntu.com/blog/canonical-releases-ubuntu-24-04-noble-numbat

These are running Ubuntu 20.04, which is EOL Apr 2025, with extended security maintenance ending another 5 years later.

ocp13 will require more care, as it's an important publicly available service. We can review the server specs as it might be over-compensating. We can also restore the SSH port to the default.

• [ ] ocp13: Need to replace gateway6: with e.g.

                - to: default
                  via: fe80::1

[jpmckinney]

This will be the first time we move the registry.

We should start the move when all jobs are complete, and after stopping all workers.

We need to move:

• /data/storage/exporter_dumps/, but it's 453G, so we should check that we're only moving active jobs. Not sure if some jobs aren't getting deleted (follow-up #312). Maybe take the opportunity to rename to exporter.

We don't need to move:

• /data/storage/kingfisher-collect/

Not sure if we need to move:

• /home/collect/scrapyd/logs (15GB). It's useful for debugging, but we could also just restart a crawl. Noting that log files are not deleted by the Collect TaskManager (probably for that reason and because they don't take up much space).

[RobHooper]

It is advisable to hold off on upgrading until Ubuntu 24.04.1 (the point release that marks this as production ready). According to the release schedule, the .1 release date is: 15 Aug 2024. This also gives third parties time to better support (and find bugs in) the new stack.

[RobHooper]

ocp13 is a CPU intensive server, this needs considering when speccing the new hardware.