microsoft / ga4gh-tes

C# implementation of the GA4GH TES API; provides distributed batch task execution on Microsoft Azure
MIT License
32 stars 26 forks source link

Move all Batch node initialization and clean-up operations to the runner #363

Open giventocode opened 1 year ago

giventocode commented 1 year ago

Overview

The BatchScheduler in the TES server generates, uploads, and manages the execution of bash scripts that perform the following operations on the execution node:

However, these operations should be performed before the execution of a container command by the TES runner.

AC:

ngambani commented 10 months ago

@giventocode can we link this issue to the TES runner https://github.com/microsoft/ga4gh-tes/issues/148? Or should this be separate from the runner implementation?

BMurri commented 8 months ago

@MattMcL4475 #515 implements Clean-up/prune Docker container images. but does not implement Move the Docker data/image directory to the data disk.

If we are going to retain the current behavior of having TES generate start tasks for the docker configuration part, then we should state that and reclose this issue. Alternately, we could open a new issue linking back to this one to move the docker configuration task from the TES server to the TES runner. Otherwise, the issue is only half completed.