As a website user
I want gigadb.org to be deployed from CI/CD pipeline onto AWS environments
So that I can work with a robust and frequently updated website
Acceptance criteria
When all this is done:
Identify peculiar datasets (which include the ones that are held up by BGI/CNGB legalities) and store on spreadsheet
Replace urls of non-peculiar datasets available from wasabi on staging (and capture how long it took)
Regenerate readme files for the datasets on staging (and capture how long it took)
Revisit how we integrate with GitLab Terraform state management to ensure any of us can reconnect the up-to-date state of what's actually deployed
Destroy and rebuild the Upstream environments, and put back bastion user credentials
Make sure provisioning script are idempotent on Upstream environments
Rehearse DB backup restoration and make sure there are no errors doing so, and that we can continue provisioning the environment afterwards
a. Must be easily restorable
b. Must still be able to provision the environment after that
c. Restore up to 1 h before incident
d. Potentially get rid of problematic methods of restoration
Get rid of Giga review pipeline jobs
Document and implement release protocol for gigadb-website web application:
a. Use semantic versioning
b. Document when to release to live
c. Use git tags to add version number to develop branch
d. Display version number in the admin interface
e. Add gigadb-website to software heritage archive
Make SOP for accessing monitoring tools (Grafana/Prometheus and Tideways)
Stop the DB synchronisation
Replace urls of non-peculiar datasets available from wasabi on live
Regenerate readme files for the datasets on live
Stop the daily backup of database done on CNGB
Secure time slot for curators to upload images to the most recent datasets and to the most popular datasets (100s)
Change the DNS record of gigadb.org to point to elastic Ip address of Upstream live web app EC2 instance
rija
commented
6 months ago
User story
Acceptance criteria
When all this is done:
Identify peculiar datasets (which include the ones that are held up by BGI/CNGB legalities) and store on spreadsheet
Replace urls of non-peculiar datasets available from wasabi on staging (and capture how long it took)
Regenerate readme files for the datasets on staging (and capture how long it took)
Revisit how we integrate with GitLab Terraform state management to ensure any of us can reconnect the up-to-date state of what's actually deployed
Destroy and rebuild the Upstream environments, and put back bastion user credentials
Make sure provisioning script are idempotent on Upstream environments
Rehearse DB backup restoration and make sure there are no errors doing so, and that we can continue provisioning the environment afterwards a. Must be easily restorable b. Must still be able to provision the environment after that c. Restore up to 1 h before incident d. Potentially get rid of problematic methods of restoration
Get rid of Giga review pipeline jobs
Document and implement release protocol for gigadb-website web application: a. Use semantic versioning b. Document when to release to live c. Use git tags to add version number to develop branch d. Display version number in the admin interface e. Add gigadb-website to software heritage archive
Make SOP for accessing monitoring tools (Grafana/Prometheus and Tideways)
Stop the DB synchronisation
Replace urls of non-peculiar datasets available from wasabi on live
Regenerate readme files for the datasets on live
Stop the daily backup of database done on CNGB
Secure time slot for curators to upload images to the most recent datasets and to the most popular datasets (100s)
Change the DNS record of gigadb.org to point to elastic Ip address of Upstream live web app EC2 instance