copasi / cloud-copasi

cloud-copasi is a web service to manage COPASI simulation tasks in HPC facilities and compute clouds, such as condor, AWS, etc.
GNU General Public License v3.0
5 stars 1 forks source link

Maybe generate cloud_copasi/settings.py during Docker build. #23

Open bdklahn opened 2 years ago

bdklahn commented 2 years ago

@hasanbaig asked, "We also have to modify settings.py file (may be we can include it in the instructions?)"

Perhaps we can change/update the settings.py.EXAMPLE to fully work with the Docker build, and then COPY it over to the container image's settings.py during the build. Or, if that is not also a decent "example" for a regular build, we could add a settings.py.DOCKER. But I like the first option better, as it is simpler, and requires less files to keep updated. I think the Docker build should be a model example for our standard installation (and thus be associated with the best "EXAMPLE").

hasanbaig commented 2 years ago

It contains a password and some secret keys which I believe cannot be made public.

And yes, settings.py.EXAMPLE has to be updated with latest contents (from current settings.py file).

bdklahn commented 2 years ago

Yes, I've thought of this, when I was looking at that Docker documentation docker-compose Django example. I think for the Docker container it might be safe to put those in there: Those might stay private to the containers and special network the docker daemon creates for them to communicate with one another. But if not, or if there are other things which might end up getting exposed through the service, we should be able to dynamically read in values from the shell environment.

In other words, it might not matter if others know the passwords the Docker containers use, if they have no way of getting in to do anything with them.

And it's already a problem, if we have been tracking (committing to this repo) a current (working) settings.py file. That should only be created upon installation, and not pushed to this repo.

hasanbaig commented 2 years ago

Current settings.py file is not a part of this repo at the moment. But we can update the settings.py.EXAMPLE file with all necessary configurations set other than private stuff. This is in my to-do list also.

Please let me know if I you want to keep in a docker file and want me to do this first?

bdklahn commented 2 years ago

Yeah . . . we can confirm, but that Docker Compose Django example seems to imply it would be safe to store passwords in there (maybe because of reasons I mentioned above). But we'd probably want to make it clear, in code comments, that they should use a different way to get secrets in (e.g. Python os.env), if they are not using the Docker container install.

hasanbaig commented 2 years ago

Okay. BTW, I am now updating settings.py.EXAMPLE file with correct file paths and other configurations.

hasanbaig commented 2 years ago

Yeah . . . we can confirm, but that Docker Compose Django example seems to imply it would be safe to store passwords in there (maybe because of reasons I mentioned above). But we'd probably want to make it clear, in code comments, that they should use a different way to get secrets in (e.g. Python os.env), if they are not using the Docker container install.

I have updated settings.py.EXAMPLE file. This can be used in place of settings.py file for cloud-copasi django project. Users need to change 1) the database password:

https://github.com/copasi/cloud-copasi/blob/134c7bb5fe20b623d2d370dc1a08586ee6c7897a/cloud_copasi/settings.py.EXAMPLE#L27

and 2) the Secret key:

https://github.com/copasi/cloud-copasi/blob/134c7bb5fe20b623d2d370dc1a08586ee6c7897a/cloud_copasi/settings.py.EXAMPLE#L33

Apart from this, they would need to set up the host and domain related information. That's it!

We can keep this file in docker as settings.py file and add instructions to make the specific changes.

bdklahn commented 2 years ago

Thanks for updating this! I like those changes.

What I was thinking was to, if possible, make settings.py.EXAMPLE exactly match what would work for the docker-compose build. Then the Dockerfile would simply COPY that into the container as settings.py.

Regardless, I think the EXAMPLE defaults should be close to a production example. (What do you think?)

Relative to that, looking through the file a little, might we want to (e.g.) change . . . DEBUG = True #Set it to FALSE for production server to something like LOG_LEVEL = 'WARN' #Set it to 'INFO' or 'DEBUG' for testing Just thoughts to avoid folks having to make a bunch of binary changes/decisions down in code messages. But maybe DEBUG is a standard variable, baked into Django?

bdklahn commented 2 years ago

Ok . . . It does also look like the settings.py example here . . . https://docs.docker.com/samples/django/ . . . uses the environment to more safely bring in the secrets (But I think the compose yaml file actually sets those up). So that looks to be the best practice for both Docker and non-Docker scenarios.

hasanbaig commented 2 years ago

Yes, this DEBUG = True or False is the standard and the file settings.py is auto generated file once the django project is launched.

The example file is very close to a production example. As I said before, all what user needs to do is to make the changes mentioned in https://github.com/copasi/cloud-copasi/issues/23#issuecomment-1059224408 along with their host/domain information.

Setting Debug = True only displays "Error traceback" on browser (so any user can see the traceback which an admin might not want to show), otherwise not if it is set to False.

hasanbaig commented 2 years ago

Ok . . . It does also look like the settings.py example here . . . https://docs.docker.com/samples/django/ . . . uses the environment to more safely bring in the secrets (But I think the compose yaml file actually sets those up). So that looks to be the best practice for both Docker and non-Docker scenarios.

Yes, but in our case, we don't need to create a django project. Whenever a django project is created, it auto generates settings.py file with a unique SECRET_KEY and all other configurations. Since, we do not need to generate a django project, therefore the SECRET_KEY in the settings.py file we put in the docker file has to be changed by a user.

bdklahn commented 2 years ago

Hmmm. I would think the auto generated settings.py might just be what happens in a single user example scenario, when they don't even have a project already. So, I would say, for our example settings.py, for our already-made project, we should have things like os.environ.get(. . .), and maybe any keys which have been generated, for the Docker container to use. Then, I would say, for anyone (still) willing to try and run this manually, there would be comments in the example for how to generate/use their own SECRET_KEY, etc. I guess I'd like to set it up so users don't have to manually edit anything in settings.py, if they use our Docker system. But leave comments in the example file for how to (as little as possible) edit their settings.py, if they don't want to use Docker (in which case, they will already also be doing a lot more, with having to set up the other dependencies, etc.).

We're probably thinking along the same lines. I just want to set it up so that, if possible, everything can just work (and still be secure best practices), without anyone having to edit settings.py, etc. It seems like that docker-compose Django example must already account for/imply this. Maybe not.

I mean, as it currently stands, our Deploy documentation basically just says 'Go edit settings.py as appropriate'.

bdklahn commented 2 years ago

So, I think, all we really might need to change to have settings.py.EXAMPLE be able to be a drop-in replacement for the eventual docker-compose settings.py, would be to change those hard-coded db entries to something more like . . .

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': os.environ.get('POSTGRES_NAME'),
        'USER': os.environ.get('POSTGRES_USER'),
        'PASSWORD': os.environ.get('POSTGRES_PASSWORD'),
        'HOST': 'db', # '127.0.0.1' or '' (empty) for localhost
        'PORT': '5432', # '5432' is the default PostgrSQL db port
    }
}

And that would also be an appropriate (more secure) example for any user wishing to try the full manual install.

hasanbaig commented 2 years ago

I agree! Lets proceed ahead with this!

bdklahn commented 2 years ago

Ok. I can push that. be9c75276152eec9920f5d94d924dedb5f0206ab 6cd1eca13a61d2f33c9885963da88480fe7343e1