Feat 47 populate metadata - Githubissues

AllenNeuralDynamics / aind-data-asset-indexer

MIT License

0 stars 0 forks source link

Feat 47 populate metadata #48

Closed jtyoung84 closed 1 month ago

jtyoung84 commented 1 month ago

Closes #47

Adds utility methods to make it easier to handle json files on s3
Adds job that will populate S3 bucket with metadata files.
Uses dask to process files in parallel
Adds lookback_days option to settings, but currently processes everything