search

Deltares / Wflow.jl

Hydrological modeling
https://deltares.github.io/Wflow.jl/dev/
MIT License
116 stars 21 forks source link

Create Dockerfile and host image publically #109

Closed evetion closed 1 year ago

evetion commented 3 years ago

Maybe as a julia tag on the current Wflow image?

visr commented 3 years ago

What would you want the Docker to contain? The main entry point is the wflow_cli app, for which a tarball can be downloaded from https://download.deltares.nl/en/download/wflow/. Is that enough? Or if you need a Julia REPL, I guess we should take the julia docker, and use PackageCompiler.jl to bake Wflow.jl into the system image for fast startup.

evetion commented 3 years ago

I don't need the REPL, I would just like to start the model from the commandline. So that's the cli app I guess? PackageCompiler.jl could be nice, but not a hard requirement I'd say.

visr commented 3 years ago

Ok yes in most cases the cli app would be sufficient, and easiest. Not sure what the best practices are for throwing a generic linux tarball into a Docker container, but that's all that needs to happen.

evetion commented 3 years ago

Manual tryout. It now runs the test toml found in the documentation from /data. You can mount your own data and the cmd takes the location of the toml file:

ENTRYPOINT [ "/opt/wflow_cli/bin/wflow_cli" ]
CMD ["/data/sbm_simple.toml"]

Now available for public testing:

❯ docker run deltares/wflow:julia
┌ Info: Run information
│   model_type = "sbm"
│   starttime = CFTime.DateTimeStandard(2000-01-01T00:00:00)
│   Δt = 86400 seconds
└   endtime = CFTime.DateTimeStandard(2000-01-10T00:00:00)
Progress:   0%|                                         |  ETA: N/A
Progress:  10%|████▏                                    |  ETA: 0:00:14
Progress:  20%|████████▎                                |  ETA: 0:00:07
Progress:  30%|████████████▎                            |  ETA: 0:00:05
Progress:  40%|████████████████▍                        |  ETA: 0:00:04
Progress:  50%|████████████████████▌                    |  ETA: 0:00:03
Progress:  60%|████████████████████████▋                |  ETA: 0:00:02
Progress:  70%|████████████████████████████▊            |  ETA: 0:00:01
Progress:  80%|████████████████████████████████▊        |  ETA: 0:00:01
Progress:  90%|████████████████████████████████████▉    |  ETA: 0:00:00
Progress: 100%|█████████████████████████████████████████| Time: 0:00:04
Progress: 100%|█████████████████████████████████████████| Time: 0:00:04
verseve commented 3 years ago

Nice! @Pieter9011 could you test this?

verseve commented 3 years ago

Suggest to use config.toml for the TOML file. Or as argument to the docker if this is possible?

evetion commented 3 years ago

You can pass the location of the toml file on the command line. You'd still have to mount a data directory with said toml and data next it, correctly referenced.

So if you've got a localdata directory with data and a .toml:

docker -v localdata:/data deltares/wflow:julia /data/config.toml

evetion commented 2 years ago

So I've updated the Dockerfile based on the code https://gitlab.com/deltares/wflow/wflow_cli and pushed it to the :julia tag. I've made sure to also make a :python tag, based on the :latest tag, that still is based on the openstreams repository.

As soon as the Dockerfile is merged into wflow_cli, Teamcity can than build a new :latest tag, next to the manual release.

In the future we might move the cli code here and automate it further.

JoostBuitink commented 1 year ago

Thanks again for picking this up @evetion!

It would also be really nice if previous versions of wflow remain accessible through their version as a tag (so wflow:v0.6.3 for example). Is it possible that every release gets two tags (latest and v0.7.1, for example)? This way latest always refers to the latest version, and you can use the version number to get a specific (older) version of Wflow. This is something that needs to be done in TeamCity I guess, right?

evetion commented 1 year ago

Sure, but Teamcity looks to the git repo for tags, and for https://gitlab.com/deltares/wflow/wflow_cli/-/tags, there are none.

I propose to merge the wflow_cli repo here, so we can make use of the releases (and thus tags) made here.

JoostBuitink commented 1 year ago

Ah right, that makes sense! We are indeed planning to merge the wflow_cli repo to here, so lets do that first!

evetion commented 1 year ago

I'm happy to update TC afterwards 👍🏻

JoostBuitink commented 1 year ago

That would be great! I have added the wflow_cli code to this branch https://github.com/Deltares/Wflow.jl/tree/merge_cli. Is there anything else that needs to be added? I see that in Ribasim.jl you have a .teamcity folder, I assume we need something like this as well?

visr commented 1 year ago

Probably best to discuss in #274 or a separate PR. I think it's worth it to copy as much as possibly, e.g. also the build/create_binaries folder, since it will make it easier to create libwflow once we'll start coupling.

Regarding the .teamcity folder, syncing TeamCity settings to the repo can be enabled from the TeamCity side, so no need to do that in your branch.