Test recovery of FOM database

[MCatherine1994]

Describe the task We can try to follow the documentation here https://github.com/bcgov/nr-old-growth/wiki/Database-Backup-and-Restore to restore the database from a backup file

Acceptance Criteria

• [x] Test the restore backup in DEV namespace
• [x] Grab the backup file from openshift
• [x] Try the restore with dev database, open a pr for that
• [x] Document the procedure

Additional context

• Make the data change, wait a day, restore it back
• Make another data change, wait a day, restore it again

[MCatherine1994]

https://github.com/bcgov/nr-fom/wiki/Database-Restore

[MCatherine1994]

I'm also not sure since for FOM we are sensitive to the date and FOM status. When we do restore that could cause FOM status change, we might need to be careful, if our cron job can pick up the FOM status change correctly.

[basilv]

So we have a process defined that only works in dev it seems. This is a start, but we need a process defined for test/prod environments more importantly than for dev. A process that only works for dev isn't enough.

I'm nervous about having a cron job scheduled for Feb 31st, is there another way we can set that up (e.g. by having no schedule at all). Does OpenShift support manually executed tasks. Why not just have yaml to create the OC job from scratch to run it, rather than creating from a cron job? See e.g. https://codergists.com/redhat/containers/openshift/2017/04/18/running-once-off-tasks-in-openshift.html

[ianliuwk1019]

I was able to follow wiki procedure for the restore process and restore fom database from the point in the past, using currently deployed openshift DEV environment fom-24 deployment.

- Use "admin" page to search all FOMs at initial status.
- Create a new FOM at initial status and search to make sure the new FOM is there
- Use Openshift console to scale admin/public/api deployment down to 0.
- Intall oc command CLI tool and login to Openshift FOM dev project.
- Skip manual backend (before restore) for now.
- Use Openshift console to edit `fom-24-db-restore` pod yaml file to supply value for `BACKUP_FILE_DIR` to the latest backup and save.
- Follow command on wiki's Restore Process session and using oc CLI tool to restore database at past point `oc create job --from=cronjob/fom-24-db-restore db-backup-restore-2024-08-28`.
- Scale admin/public/api up to 1 pod and verify by searching on admin page for FOMs with initial status and make sure the new FOM previously created did not show up after restore process.

I also adjust the wiki process to include a step for how to install oc CLI command tool locally since initially I had no tool to oc command and had little bit trouble. @basilv