crkn-rcdr / cihm-metadatabus

Documentation and Docker build environment for key portions of the metadata bus.
BSD 2-Clause "Simplified" License
0 stars 1 forks source link

ocrtask: Support "resume" for download. #57

Closed RussellMcOrmond closed 9 months ago

RussellMcOrmond commented 2 years ago

Currently if a download gets interrupted or fails, there will be a failure creating the directory a second time.

Example:

2022/09/21 16:14:53 - ERROR {CIHM.TDR} [CIHM::Meta::Ocrtask::catch {...} ] 6fd75576ee099b1528fa950b0421235e: Can't create task work directory /home/tdr/ocr/Batch_485_Heritage : File exists

John has indicated that batch names will always be unique, so we can assume that if a directory already exists that the operator is intending to resume. The tool should no longer fail on creating the directory, and should check if a file that already exists on the filesystem is the correct one before downloading again.