Allow for multiple projects to be run on the same host

[chiruzzimarco]

Currently derex have serious issues in running multiple projects (or multiple environments of the same project) on the same host since services are on a per-host basis. The whole reasoning behind the ddc-services command was about having per-host services (a single mysql, rabbitmq, mongodb, elasticsearch serving any project) in order to speed up development time and do not clobber a developer machine working on a lot of different projects. This multitenant behaviour was implemented gradually by:

• naming mysql and mongodb databases after the project name
• naming rabbitmq queue after the project name
• naming docker containers after the project name

This is all nice, except that some issues still persists and others may be unsolvable:

• the elasticsearch indices should be named after the project name. This involve changing a lot of Openedx Django settings and may even prove impossible without changing some core code
• since ddc-services is run on a per-host basis it won't account for any particular project. Thus it can't know that a specific project is running Openedx on juniper release and needs mysql 5.6 while a different one is running Openedx on koa release and needs mysql 5.7
• different environments of the same project cannot run on the same host without changing the project name since they will conflict and overlap. The need to change the project name in order to run the same project twice (production/staging) is conceptually wrong.
• even if it was in the plan to hash them with the project name, secrets are stille derived from the host main secret only. Thus different projects will share secrets
• derex mysql and mongodb commands currently act with root privileges in order to be able to create/drop databases and users. This means that a project is authorized to drop another project related database/user. Same applies to rabbitmq queues and MinIO buckets.
• every project belongs to the same network and use the same internal httpserver. Thus containers from different projects are able to communicate between each other.

For those reasons i suggest removing ddc-services altogether and include all services on a per-project basis. The only case where i see it fit is about host monitoring (still to be implemented).

[chiruzzimarco]

Elasticsearch indexes names are hardcoded in the edx-platform code here:

https://github.com/openedx/edx-platform/blob/master/cms/djangoapps/contentstore/courseware_index.py#L343 https://github.com/openedx/edx-platform/blob/master/cms/djangoapps/contentstore/courseware_index.py#L449 https://github.com/openedx/edx-platform/blob/master/cms/djangoapps/contentstore/courseware_index.py#L554

If there are other locations, i was not able to find them. I suggest we replace the courseware_index.py module with a version which tries to take the elasticsearch index name from Django settings and defaults to the original name if the setting cannot be found.

Each supported openedx release needs to have it's own module version customized here.

This will allow us to namespace the index name like we are already doing for the mysql database name for example.

This should be handled in a separate PR so to restrict the scope of this PR. It will be useful regardless of the choices we will take on derex future developments about handling different project environments.