feat: do not copy to /original_metadata if already exists - Githubissues

AllenNeuralDynamics / aind-data-asset-indexer

MIT License

0 stars 0 forks source link

feat: do not copy to /original_metadata if already exists #69

Closed helen-m-lin closed 3 weeks ago

helen-m-lin commented 4 weeks ago

closes #62 In both the populate and index jobs, always check if /original_metadata exists so to not overwrite any files accidentally.

does_s3_metadata_copy_exist checks for files that conform to /original_metadata/{core_schema}.{datestamp}.json
If any such file exists, do not copy to /original_metadata
refactored to remove some redundant code