add `medienhaus-cms` and `medienhaus-api` options to composition

[andirueckel]

This adds medienhaus-api and medienhaus-cms to the medienhaus/ docker composition.

As medienhaus-api depends on a valid matrix-synapse account to manage spaces and rooms, we need to register such an account with matrix-synapse before starting up; this splits the .env variable substitution as well as the start-up of the docker composition in two phases; this pull-request adds the init.sh helper script to aid with the second phase.

The init.sh script is non-repeatable, only executed once for registering the medienhaus-admin account with matrix-synapse and retrieving an access_token and writing it to the .env file. The init.sh script could be re-run after changing MEDIENHAUS_ADMIN_USER_ID in .env, and thereby registering a new account with matrix-synapse.

The envsubst.sh script is repeatable; every time changes are made to .env or the template/* files, re-run the script and re-create the configuration files and docker compose up -d --force-recreate --build --no-deps …

Please consult both scripts’ help/information regarding the different command line arguments in case you’re interested in including medienhaus-api and/or medienhaus-cms via the following commands:

sh ./scripts/init.sh --help

sh ./scripts/envsubst.sh --help

Note: The etherpad-lite submodule is pinned at tags/v1.9.1 because of a newly introduced bug in the ether/ueberDB dependency, which was fixed by @robertschnuell in https://github.com/ether/ueberDB/pull/464.

[andirueckel]

@marcel-klasse @robertschnuell please test this branch locally by cloning the repository to a new folder on your computing machines and following the steps in the README as if you’d never have read it before. 👀

@fnwbr In case you, too, want to test this branch before merge …

[fnwbr]

Also fixes https://github.com/medienhaus/medienhaus-docker/issues/6 by introducing the template/etherpad-myapds.configuration.js template and mounting it over the original file ep_mypads/configuration.js inside the etherpad docker container; it works, for now.

How is this better than inserting to rows into the Postgres database? What if ep_mypads changes something in their configuration.js? Will we have to adjust our template/X then? and not just the template: will all users of medienhaus-docker, given they have turned their template stubs into actual config files, have to re-generate them?

Like you said ("it works, for now") this approach feels very brittle to me; while not seeing any downsides of booting the Etherpads Postgres database with the corresponding values filled at first start.

[robertschnuell]

Also fixes #6 by introducing the template/etherpad-myapds.configuration.js template and mounting it over the original file ep_mypads/configuration.js inside the etherpad docker container; it works, for now.

How is this better than inserting to rows into the Postgres database? What if ep_mypads changes something in their configuration.js? Will we have to adjust our template/X then? and not just the template: will all users of medienhaus-docker, given they have turned their template stubs into actual config files, have to re-generate them?

Like you said ("it works, for now") this approach feels very brittle to me; while not seeing any downsides of booting the Etherpads Postgres database with the corresponding values filled at first start.

temporaily replacing the configuration.js with an a template one by the docker composition is not meant as permernent. A PR (https://framagit.org/framasoft/Etherpad/ep_mypads/-/merge_requests/119) for mypads is alreay in place from my side. Just have to discuess the issue a bit with the mypads maintainers. If this got merged there is no need anymore to mount a custom configuration.js

[fnwbr]

🤔 ... and where will LDAP settings be defined then?

[robertschnuell]

🤔 ... and where will LDAP settings be defined then?

in the config/etherpad.json based on template/etherpad.json with the ep_mypads key which holds an object with the configuration settings for the mypads plugin. etherpad would ignore this specific key but the ep_mypads plugin got access to the global etherpad configuration file as it is imported already in the configuration.js file of ep_mypads: https://framagit.org/framasoft/Etherpad/ep_mypads/-/blob/master/configuration.js?ref_type=heads#L32 but as it is still needed that our PR got accepted by the ep_mypads maintainers, this isnt possible for now to set all ep_mypads plugin configurations via etherpad.json.

[fnwbr]

Ahhh, I see. 👍 Let's see what they have to say...

[andirueckel]

How is this better than inserting to rows into the Postgres database? […] Like you said ("it works, for now") this approach feels very brittle to me; while not seeing any downsides of booting the Etherpads Postgres database with the corresponding values filled at first start.

@fnwbr From what I know, Etherpad discourages working directly in their database. If you’d want to experiment on this together, let me know, but I myself am also not keen on manually writing to the database; _also what if ep_mypads changes how/what/where they store in the database?_

What if ep_mypads changes something in their configuration.js? Will we have to adjust our template/X then? and not just the template: will all users of medienhaus-docker, given they have turned their template stubs into actual config files, have to re-generate them?

@fnwbr Well, this is the case for almost each and every one of the template, i.e. config files; and it’s also true for each and every one of our dependency/submodule version. Things will develop, things will needs updates, things will break, things will need fixes; and the fixes part will be especially relevant for custom things we want the dependencies/submodules to support.

temporaily replacing the configuration.js with an a template one by the docker composition is not meant as permernent. A PR (https://framagit.org/framasoft/Etherpad/ep_mypads/-/merge_requests/119) for mypads is alreay in place from my side. Just have to discuess the issue a bit with the mypads maintainers. If this got merged there is no need anymore to mount a custom configuration.js

@robertschnuell Well, let’s see—I’m always happy about making all of this easier and less complex. As written above and highlighted by @fnwbr, it works for now, and that’s what counts for me in regards to this development-first repository.

[fnwbr]

@andirueckel

Etherpad discourages working directly in their database. If you’d want to experiment on this together, let me know, but I myself am also not keen on manually writing to the database

The configuration file you linked to is literally mentioning that it supports and respects data, that it finds in the database: https://framagit.org/framasoft/Etherpad/ep_mypads/-/blob/master/configuration.js?ref_type=heads#L219 https://framagit.org/framasoft/Etherpad/ep_mypads/-/blob/master/configuration.js?ref_type=heads#L78-79

Regarding on how to insert data into the database at init time: https://github.com/docker-library/docs/blob/master/postgres/README.md#initialization-scripts

So basically, instead of echoing the contents of the template into config/etherpad-mypads-ldap-configuration.json you'd just echo it into sql/etherpad-mypads-ldap-configuration.sql, mount that file under /docker-entrypoint-initdb.d/x.sql for the postgres-etherpad container, and last but not least, make sure that it's valid SQL instead of just JSON: (not tested)

INSERT INTO store (key, value) VALUES ('mypads:conf:authLdapSettings', '{<... JSON ...>}');

[andirueckel]

🤔 ... and where will LDAP settings be defined then?

in the config/etherpad.json based on template/etherpad.json with the ep_mypads key which holds an object with the configuration settings for the mypads plugin. etherpad would ignore this specific key but the ep_mypads plugin got access to the global etherpad configuration file as it is imported already in the configuration.js file of ep_mypads: https://framagit.org/framasoft/Etherpad/ep_mypads/-/blob/master/configuration.js?ref_type=heads#L32

@robertschnuell Oh yes, we evaluated that, but I forgot to implement this.

@fnwbr This might even remove the need to modify any ep_mypads specific configuration file, and/or database.

EDIT: I stand corrected, we still want to configure the HTMLExtraHead option with all of the <script>…</script> stuff for styling the ep_mypads password prompt; or maybe we could inject css/js via medienhaus-spaces as we already do for the iframed element-web.

b96620c2fa53c22825a97cb2a076dc1eb7d9faf2