broadinstitute / gdctools

Python and UNIX CLI utilities to simplify interaction with the NIH/NCI Genomics Data Commons
Other
31 stars 4 forks source link

change when ingestor syncs data & writes VERSION to cloud #80

Open noblem opened 6 years ago

noblem commented 6 years ago

Updating GDAC data in the cloud depends only upon a successful dicing: it does not require a loadfile or sample report to be generated. Because of this, after a successful dicing the ingestor should sync the dicing to the cloud & update the cloud data version BEFORE attempting to generate loadfile & sample report. This decreases the # of moving parts that must work properly to keep the data in cloud synched, and thus increases the robustness & likelihood of cloud data being as current as possible.