Wrong directory permission on deployment

[eboileau]

A clear and concise description of what the bug is.

On deployment, __create_local_folders.py is run, and directories under HOST_DATA_DIR end up with the wrong ownership. This is "commented out" in the installation playbook. mini_cron.sh should also be more forgiving.

This can be fixed after install, but ideally we want a clean install.

[eboileau]

We now have conflicting directory creation between create_local_folders (container) and FileService (both dev and prod): this should happen only once, probably by FileService, and be safe (create if not exists, i.e. exists_ok=True).

[eboileau]

The only tricky one is IMPORT_DIR, which should exists on startup... see also #126

[HaraldWilhelmi]

As discussed with Etienne:

• Whenever possible SCI-MODOM should create missing directories. Setting them up is the responsibility of the FileService. That should be done by now.
• The little housekeeping job (mini-cron.sh) running in the container outside of Flask should silently ignore directories, which have not been created yet. This is a TODO.

[HaraldWilhelmi]

Etienne pointed out that we should also move out as much as possible from docker/scripts/__create_local_folders.py. What needs to stay?

• Everything needed by the docker-compose.yml (HOST_SECRETS_DIR, HOST_IMPORT_DIR, HOST_DATA_DIR, HOST_BACKUP_DIR, HOST_CONFIG_DIR, HOST_DB_DATA_DIR)
• Everything needed by the script itself (HOST_SECRETS_DIR, HOST_CONFIG_DIR)

[HaraldWilhelmi]

Permission setup:

• Host user is mapped to root (UID 0) inside the container.
• Host user's primary group is mapped to group root (GID 0) inside the container.
• The data of user 'app' is not accessible from the outside of the container (except for root).

Possible approach:

• Use inside the container GID 0 with umask 007, and setgid on directories.
• The app user has access thanks to being the owner.
• From outside the files are accessible via the group's permission.

To do that, we need:

• Change permissions to GID 0 for pretty much everything in the container on startup.
• Use setgid bit on directories.
• Tell the file service to use umask 007.