Improve local deployment

alan-turing-institute / DTBase

A starting point from which digital twins can be developed.

MIT License

11 stars 4 forks source link

Improve local deployment #235

Closed EdwinB12 closed 6 months ago

EdwinB12 commented 7 months ago

This PR wraps up the deployment of dtbase into a single docker-compose.yaml .

Things still to do:

Sort out Volumes. Seems to currently work but would rather you explicitly define it.
Understand how this works for developers.
Consider freezing Postgres version
Reduce Image Size
Move dockerfiles into a folder to clean up top level directory.

EdwinB12 commented 6 months ago

Clean up handling environment variables
remove sourcing of environment variables from run.sh files
remove run_localdb. Delete line 77 in pyproject.toml. Can delete main function in dtbase/backend/database/docker.py
How to develop whilst running these containers? rebuilding the container is a not valid option.
Documentation

EdwinB12 commented 6 months ago

Hey Markus, PR for docker deployment.

I've done a fair amount of restructuring on the developer docs readme so there will likely be clashes with your documentation branch?

I did merge your docs branch into this one at some point so there shouldn't be too many conflicts I hope. .

Offline for a lot of today, but happy to go through this PR on Friday if that works for you.

mhauru commented 6 months ago

I tried running locally and got No space left on device when the db container tries to create the database, will need to figure out what went wrong there.

Nevermind that, had some issue with my Docker setup, fixed up by an upgrade and rerun. Works well now, great work @EdwinB12!

EdwinB12 commented 6 months ago

resolve #146 #218 #157

EdwinB12 commented 6 months ago

Do you think there's could be a way to simplify the secrets management so that we wouldn't have both the shell script and the .env file? Having both might confuse a lot of people, and seems redundant. I would be happy to e.g. switch using .env for everything and get ride of the .secrets folder.

I did look at this. Unfortuantely, docker REQUIRES the .env file to be in the same directory as the compose file. In terms of replacing the .sh file, its not as clean as you like. For instance, you can't simply source the file. Discussion here: https://gist.github.com/mihow/9c7f559807069a03e302605691f85572?permalink_comment_id=3731592

EdwinB12 commented 6 months ago

Actually, this was a bad idea because the file in under version control. People will accidentally end of commiting. I'll rename it back to default.env. Instructions are in the Developer Docs that explains the user needs to rename it.

EdwinB12 commented 6 months ago

This was a messy merge, don't think I messed up any of the conflictsbut its possible.