search

microbiomedata / nmdc_automation

Prototype automation
2 stars 2 forks source link

Add md5sum checks for data copied to results folder #192

Open aclum opened 4 months ago

aclum commented 4 months ago

Update automation code to check the md5sum values to make sure file transfers to /global/cfs/cdirs/m3408/results worked correctly. The data_object_set md5_checksum values can be used for this.

Example where this didn't work properly https://github.com/microbiomedata/issues/issues/716

aclum commented 2 months ago

FWIW JAWS is using shutil.copy + filecmp.cmp, said this is less intensive than md5sum.