microsoft / ga4gh-tes

C# implementation of the GA4GH TES API; provides distributed batch task execution on Microsoft Azure
MIT License
34 stars 27 forks source link

[#627 expired SAS tokens] Call the runner from the start task #631

Open BMurri opened 8 months ago

BMurri commented 8 months ago

Problem: The current method of managing "global" start tasks set in configuration, compute node image-based start-task commands used to configure the docker daemon service, and future per-vmfamily start-task activities (including configurable/installable ones as envisioned for Illumina DRAGEN support) is really brittle.

Solution: The runner should be able to run multiple scripts. Consider whether to extend the current task JSON or to instead create a new schema for start tasks. Then, supply the start-task's JSON's (in whatever form it ends up in) URL in the start task command-line along with the bash/shell commands to download the runner's binary and chmod it. Consider building into the runner start-task tasks that we know will likely be permanent (such as configuring the docker daemon). Also consider uploading the start task's trace files (stderr.txt & stdout.txt to an appropriate location in the tes-internals container space.