The Dashboard relies on several VMs running on Wikimedia Cloud:
Programs & Events Dashboard database server
This runs MariaDB, and is configured to provide access to the database from other servers in Wikimedia Cloud. The data itself is stored on an attached Cinder volume.
Programs & Events Dashboard Sidekiq
This runs some sidekiq workers, including the one responsible for the 'long update' queue which processes the most resource-intensive course updates.
Programs & Events Dashboard Web
This runs a lot of things (too many):
The Passenger process and Apache webserver that handle web requests.
A number of Sidekiq workers
The Redis database that powers the Sidekiq job system
The non-database non-code assets, including profile pictures and the temporary CSV files that are used for data downloads and other things.
Database replica endpoint
The service that makes queries to the Replica database and returns Article and Revision data. (This is used by both Programs & Events Dashboard as well as Wiki Education Dashboard.)
Server replacement
We periodically need to replace old VMs with newer ones running supported up-to-date versions of Debian. Currently, that's very tedious because we have not automated the server setup process or moved all the data to Cinder volumes that can be re-used by new VMs.
We should:
Move Redis persistence to a Cinder volume
Create a new (small) VM just for Redis
Move the non-code non-database assets to a Cinder volume
Create a new (large) VM for all the Sidekiq workers that currently share a VM with the web service
Create automation scripts that can efficiently set up fresh VMs for each of the ones we need (web, redis, default sidekiq, large-job sidekiq, database, replica endpoint)
The Dashboard relies on several VMs running on Wikimedia Cloud:
Programs & Events Dashboard database server
This runs MariaDB, and is configured to provide access to the database from other servers in Wikimedia Cloud. The data itself is stored on an attached Cinder volume.
Programs & Events Dashboard Sidekiq
This runs some sidekiq workers, including the one responsible for the 'long update' queue which processes the most resource-intensive course updates.
Programs & Events Dashboard Web
This runs a lot of things (too many):
Database replica endpoint
The service that makes queries to the Replica database and returns Article and Revision data. (This is used by both Programs & Events Dashboard as well as Wiki Education Dashboard.)
Server replacement
We periodically need to replace old VMs with newer ones running supported up-to-date versions of Debian. Currently, that's very tedious because we have not automated the server setup process or moved all the data to Cinder volumes that can be re-used by new VMs.
We should: