mongodb docker setup missing

[megies]

@Jollyfant, any reason why the MongoDB dockerfile / docker setup is not included here?

What's your experiences with using the docker setup for WFCatalog in general? Is it advisable?

[Jollyfant]

We experimented with Docker as a way to easily deploy software across nodes. It's quite powerful, especially for the webservice IMHO but we don't use this in production at all. When you use Docker for the collection and storing of the quality metrics you will need to consider data persistence throughout container startups and wind downs. And this became more complicated than installing the software outside of the container. That of course is manageable, but it requires knowledge and goes against the philosophy of being easily deployable.. so I don't advise using the Docker images unless you are familiar with Docker.

The docker image for MongoDB was using the official one and therefore not included.

[megies]

Alright, so the docker setup was just some sandbox experiment, I guess. Thanks for the clarification.

[petrrr]

@megies: We are using a dockerized installation and we plan to do so for production as well. But I agree with @Jollyfant that some more work is required.

[Jollyfant]

So @petrrr, do you need some help with the Docker installation or is it working?

[megies]

@megies: We are using a dockerized installation and we plan to do so for production as well.

Hmm, interesting. I'm still not sure which way to go.. maybe a hybrid approach might make a lot of sense? The collector/mongoDB could probably be done on the host machine itself, so that there's no additional maintenance load on ensuring data persistence and the DB can easily be backed up with a regular cron job. On the other hand, having the exposed WS inside a closed docker image (no need for persistence there, I assume? It just needs read access to the DB? Or am I missing something..?) that can just be restarted when something goes wrong and that can not negatively affect the host system might make sense..

Any opinions on that?

[Jollyfant]

Yeah, you need to expose a port and have access to the DB but that is about it. The only issue that needs to be solved is the logging, because by default that is stored inside the container too.

[massimo1962]

about data persistence is sufficient use mounted volumes (to mounting host directory in a container); and this it is valid for all stuff (data; log; files; etc..) and you will have all the data outside the containers... e.g. docker run -p 27017:27017 -v /PATH/TO/DATASTORE:/data/db -d --name mongo_server /mongo_server:3.2 in this command we have all data DB into directory /data/db on host machine. Likely we should write better the steps for production-ready docker installation.

[megies]

Likely we should write better the steps for production-ready docker installation.

That would be really nice. The non-docker set up is also a bit hard to follow. I think there should be less focus on custom usage (like with all the examples showing python code fragments, but a stronger emphasis should be on step-by-step instructions for a "default"/"normal use case" set-up. The instructions still have me wondering e.g. in which step the mongo db tables are set up (I'm guessing the tables are set up automatically at some point, but then again there's some more mongo db commands about using some index fields).

Btw. @Jollyfant I've sent a registry request for https://dev.knmi.nl/projects/eida/wiki/WFCatalog as I'm involved with the set up at LMU. You're still at KNMI, right? It would be awesome if you maybe can check if my request can be approved or declined.

[Jollyfant]

@megies - MongoDB is schemaless so you don't set up tables. You just need to make sure it is installed and running, then configure the Collector to connect to the host:port where it is running.

I've requested to approve your registry with the IT dept.

[megies]

Thanks @Jollyfant! Both for clarification on mongoDB and the KNMI wiki registry.. 🚀