cylc / cylc-flow

Cylc: a workflow engine for cycling systems.
https://cylc.github.io
GNU General Public License v3.0
325 stars 90 forks source link

Buit-in container support #5157

Open hjoliver opened 1 year ago

hjoliver commented 1 year ago

Some users have suggested Cylc needs this capability built in (of course you could roll your own with task scripting, but many users don't know how to do it):

This task should pull down this container, and run this application in it.

(This sort of thing is becoming quite important in the wider world, of course, and even on HPC beyond atmospheric science).

oliver-sanders commented 1 year ago

Or more generally built-in environment support (Conda, Docker, Apptainer, modules, etc) as well as integration with cloud systems.

Here's an example I put together for building Conda environments. It would be good to have something more automated. A couple of considerations...

Pull or build:

Local or remote: