[728-1] Runner calculates MD5 checksum for uploads. - Githubissues

microsoft / ga4gh-tes

C# implementation of the GA4GH TES API; provides distributed batch task execution on Microsoft Azure

MIT License

32 stars 26 forks source link

[728-1] Runner calculates MD5 checksum for uploads. #729

Closed giventocode closed 2 months ago

giventocode commented 2 months ago

Applies to #728

In this PR:

Added a new property to the node task: RuntimeOptions.SetContentMd5OnUpload. When set to true, the runner will compute the MD5 for the files before the block list is committed.
This change lays the groundwork for implementing hash validation when downloading data using DRS.
By default, the setting is false since calculating the MD5 has a performance impact.

Pending Work (to complete: #728):

Expose this option in the TES server as a runtime configuration, with a default value of false.
Modify the CoA deployer to include this new configuration option.