Restore data to database instance running in Docker container

[SPlanzer]

@strk perhaps this is something you can help with.

The DB docker container requires a method for restoring data into the gazetteer database

I have done a custom (~300mb) and plain (~800mb) data only dump of the prd gazetteer DB. I will send you a presigned links to these in the projects slack channel

* Note some data is installed into the DB via /src/sql/gazetteer_sysdata.sql

For testing we may use a subset or just manually create test data. However for UAT a full dataset installed into the docker DB container will be useful.

[strk]

Do you also need a way to create such dump ? Because install_web.sh does create a gaz_web_config_dump_local.202004091337.psql file (with appropriate timestamp) whenever it is called with the drop argument against a database which already has a configuration. Do we need something similar for other data ? Or how do we define what is the data we want to restore ?

Restoring the data would require creating a user with login capability under the gazadmin role. Do you want the data loaded in the docker image or a script to load them in a volatile container ?

[SPlanzer]

Do you also need a way to create such dump ? Because install_web.sh does create a gaz_web_config_dump_local.202004091337.psql file (with appropriate timestamp) whenever it is called with the drop argument against a database which already has a configuration. Do we need something similar for other data ? Or how do we define what is the data we want to restore ?

Restoring the data would require creating a user with login capability under the gazadmin role. Do you want the data loaded in the docker image or a script to load them in a volatile container ?

@strk thanks for the response. This seems like a fair bit of work(?) I wonder if this should be on hold until we fully understand the priorities. To me this is heavily tied to how UAT will be undertaken and in what environment. Though in saying that, for me, data in the DB is definitely a "nice to have"

There are three reasons for having data restored into the docker container

• Developers can interact with the app and understand how it works. LOW Priority can just get on with writing tests based on the UAT test plan.
• Integrations tests can be written using this data - Priority not 100% clear. Integration test data approach needs to be considered. The Benefits of using the test suite (pyTest) to create data in the DB on the fly as need, removing the need for restoring data to the DB prior to the running of tests should be considered.
• A UAT database is required for the tester to connect to and test the app - Priority depends on the below.
  • Two options exist. a. the prd database is replicated to the datacom tst database server for UAT. Note; There are no automated process for establishing this test database. b. The UAT tester stands up the docker database container, data is installed into / along side the container to allow testing of the application from a UAT perspective. Note, the UAT tester will need to install docker on their machine or we will need to install the container somewhere they can access it and support port forwarding to their environment.

[strk]

For the moment, in this branch, there's a user gazadmin with password gazadmin that can be used to load data.

[strk]

I filed PR #91 to document how data can be loaded, at least as a stub. It's based on the 2to3 branch

[SPlanzer]

tests specific issue now managing this (#105)