To use PostgreSQL/YesSQL persistence, a dev should be able to use an appropriate Docker container

[craigfowler]

As suggested in #846 - we would like to add direct support for a PostgreSQL Docker container which may be integrated with the Elsa Docker container. This means that it will be easier to set up an Elsa-with-YesSQL-and-PostgreSQL-persistence environment using Docker.

This issue is to make whatever changes to the docker compose file in order to accomplish that.

[craigfowler]

Hi @sfmskywalker and @jwillmer - it seems you've done most of the work on Docker so far, I'd like your opinions since I've never actually used Docker myself yet, only read about it in theory.

Do we have some redundant/obsolete config?

Firstly am I right in thinking that the Docker compose file in the root of the repo is actually unused? Could it be safely deleted? How about the Dockerfile in the same directory?

Should we use Docker Compose Profiles to manage these dependencies?

Secondly, dependent services like PostgreSQL are elective because the end user will only actually want one out of a few available. For example when it comes to persistence, they really only want one of:

• MongoDB
• SQLite + YesSQL
  • There's not much for Docker to do here, except map up some storage for the db file
• Postgres + YesSQL
• MySQL + YesSQL
• Or one of the other possibilities

I was having a poke around Docker docs and I stumbled across Profiles. Is that the correct way to deal with elective dependencies/services like this? Would you be happy if we went down this route for those elective dependencies such as persistence backends?

I think a consequence of profiles would mean that anybody wanting to start the app from Docker would need to specify at least one of those elective profiles or else they get no storage services at all. That's not 100% app-breaking since Elsa will use in-memory persistence if no other persistence is available/configured, but it might not form a very useful demo app (restart it and all your workflows are gone). I went hunting for a way for Docker to "Automatically activate profile X if no other profiles were activated" but I don't think such a feature exists.

Perhaps we could mitigate that if we were to always do the work for SQLite in the core, so that if no other backend is configured, we can do SQLite persistence? Then, if the consumer chooses something different, all we really did is map some file storage that won't be used.

Any other thoughts on this? Really I'm just thinking about how optional (technically elective) service dependencies should work under Docker.

[jwillmer]

Do we have some redundant/obsolete config?

I never touched it. I guess it is used to spin up services that you can connect to with elsa for testing. It does not interact in any way with elsa. As long as else appsettings is not configured to use them.

Should we use Docker Compose Profiles to manage these dependencies?

I'm waiting for some configuration option in else to define the storage provider via appsettings. progress is tracked in this issue.

That sad. The way many other projects do it is the following - example pics are from Nextcloud:

• docker-compose files are always only examples. For any production deployment you need to create one yourself with revers proxy etc.
• default (example) docker-compose file uses internal storage (normally SQLite) and docker-compose specifies a volume there the content of the database is stored so that the data is persistent even if you destroy/recreate the container. In the below picture just replace mariadb with elsa to get a feeling of how that would look like. Note that no other configuration is required. Only pointing a volume to some location there the image is storing it's data.

• other storage provides can be defined by setting environment variables. This should be documented!

Example docker-compose of a service with database:

[craigfowler]

Excellent @jwillmer - thanks for that. Always good to get an opinion from someone who has some experience & knows what they're talking about 😆

On the possibly-redundant config - @sfmskywalker are you using it for anything or was it just for trying out? It's not been touched in a while and I wonder if it actually works 🤔

As for the way to wire up elective services that sounds good @jwillmer. In that case (in the scope of this issue alone), would you both agree that "the right thing to do" would really just be to provide an example docker compose file of what Elsa + Postgres + YesSQL would look like. But we don't actually build or integrate that into our "main"/published Docker projects, or actually do anything with it at all. It's just there as a sample?

[jwillmer]

Exactly. We provide a sample configuration file. But the default - running the docker image without any env. var - will be using SQLite. And we will also promote the default docker image for users to quickly spin up an instance and test it.

[sfmskywalker]

Moving this out of the V2 milestone.

[jwillmer]

@sfmskywalker how is the timeline on this? I have some users that like to try elsa and give feedback but they wont play around as long as there work is gone after a redeploy.

[sfmskywalker]

@jwillmer I created #1032 to track the development of feature-enablement via settings. Will work on it soon.

[sfmskywalker]

@craigfowler @jwillmer

On the possibly-redundant config - @sfmskywalker are you using it for anything or was it just for trying out? It's not been touched in a while and I wonder if it actually works 🤔

It's like Jens said, it's there for testing things. For example, when running the integration tests, I first startup MongoDB and Postgres containers using this file. OR when I'm testing SendMail activities, I use it to quickly start Smtp4Dev.

It's not used anywhere else and I could actually keep it separate from the project. It's just convenient to use together with Rider because it allows me to start services like this:

[jwillmer]

Appsettings:

"PersistantProviderUri": "mongodb://[username:password@]host1[:port1][,...hostN[:portN]][/[defaultauthdb][?options]]",

Postgres: https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING Mongo: https://docs.mongodb.com/manual/reference/connection-string/