DUNE / dist-comp

Action items for DUNE distributed computing, and common scripts that are used.
2 stars 0 forks source link

Get jobscripts from GitHub too #152

Closed Andrew-McNab-UK closed 5 months ago

Andrew-McNab-UK commented 8 months ago

Currently when creating a stage in a workflow you can specify the required jobscript either as a local file to upload, or as a jobscript in the Jobscript Library in the justIN database. This feature will add a third option to cause justIN to fetch the jobscript using a path in a GitHub repo at a given tag or revision hash.

This should be very straightforward to implement because raw files referenced by tag or revision hash have simple URLs like this: https://raw.githubusercontent.com/DUNE/dune-justin/906b1c226c9f579819c9e887916a1c072f84df1a/testing/hello-world.jobscript The same URL form is used for human-readable tags.

The option can be like

--jobscript-github 906b1c226c9f579819c9e887916a1c072f84df1a:DUNE/dune-justin/testing/hello-world.jobscript

As tags cannot contain colons and hashes do not, this form is safe.