Epic 8: fix route to production

phraenquex commented 2 years ago

Restore-backups-to-staging ticket
Pre-staging front/backend stack (?)

Questions:

policy on backups
handling snapshots

phraenquex commented 2 years ago

Sequence:

backup postgres
backup media files
backup discourse

Backup policy

rsync hourly to a specific place - that's what gets moved to staging. tgzip the rsync destination daily (midnight?) - 10 days is enough.

Diskspace

Please don't worry about it. Says @phraenquex

How to handle discourse

Are discourse URLs relative paths? (backend and frontend) @boriskovar-m2ms says: Currently absolute - but easy to change to relative. @duncanpeacock - check if quick DB editing.

[ ] create ticket @duncanpeacock

phraenquex commented 2 years ago

@duncanpeacock when checking #774, the data are not identical - some snapshot ID was changed.

Staging database should be stepped on completely by production database.

1) Implement as GitHub action: both Build and Sync, both manual (not auto). Action for Alan.C 2) Rerun database sync, and check #774 again.

phraenquex commented 2 years ago

Alan suggested on Slack - please do this dry-run for rel 2.6.7

Does this “process” satisfy the staging/pre-production testing needs…? I think everything is in place to do this. We don’t need any new GitHub Actions to deploy to staging, instead I think we just need to adopt this as a process. The main advantage is that it gets us into the habit of tagging pre-production builds (rather than relying on ‘latest’). So - to test a pre-production image you a) tag the staging stack with a pre-production tag ([semver-2](https://semver.org/)). It can be anything EXCEPT “N.N.N”. We tend to use things like “1.0.0-rc.1” for a release candidate for example Then b) you decide whether that image needs a copy of the latest production data (a SYNC). If not you run one set of AWX jobs, if it does you run another set of AWX Jobs. We might even be able to combine the START STOP with one Job. But the important part of this process is “there’s no code to write, no actions to develop”. I think it’s all ready to use. Regardless … I think we need to start tagging the stack for pre-production tests and (if we’re not already doing so, and we can) also tag the backend and frontend repos. A production image should be based on tagged versions of these rather than latest anyway. If the above is acceptable we could give it a dry-run at the next opportunity.

phraenquex commented 2 years ago

Three suggestions / requests from me:

that decision point in your flow-chart - add yes/no
post the flow-chart somewhere suitably visible, along with step-by-step instructions
[anal] should AWX job have a more descriptive name? E.g. Replicate production data on Staging Stack (instead of Staging Stack (Production Replica) as it is currently called)

alanbchristie commented 2 years ago

A draft of the documentation (located in the dls-fragalysis-stack-kubernetes repository) has now been released. The relevant section (Pre-release testing) can be found on the ReadTheDocs at: -

https://im-dls-fragalysis-stack-kubernetes.readthedocs.io/en/latest/development/pre-release-testing.html#

phraenquex commented 2 years ago

Thanks! @rsanchezgarc please proofread/comment, and once you're happy, reply here and move the ticket to the "Done" swimlane.

rsanchezgarc commented 2 years ago

@alanbchristie Sounds reasonable. Only one question. Do we remove the pre-release tags once we actually do the release?

rsanchezgarc commented 2 years ago

@alanbchristie any commet on that?

alanbchristie commented 2 years ago

No - just leave tags assigned. Always a little unwise to remove (or even re-use) tags. Images based on them may be around for a while. Asa general policy consider tags as write-once, read only.

m2ms / fragalysis-frontend