chanzuckerberg / cryoet-data-portal-backend

CryoET Data Portal API server & ingestion scripts
MIT License
1 stars 2 forks source link

S3 data validation: deposition, dataset, and key_images (tomogram key photos) #195

Closed daniel-ji closed 1 month ago

daniel-ji commented 1 month ago

To be reviewed and merged after #193

So as of now, deposition validation is just the deposition_metadata.json file and relevant photos, not any other actual related deposition content. Any deposition that is referenced anywhere in the dataset (annotations, tomograms, metadata, etc.) that is currently being validated will be validated alongside. As other validations are implemented (tiltseries, frames, whatnot), the validation for those fields' corresponding deposition will be added alongside (so those will be done in their respective PRs, not done in here).

Validate dataset_metadata.json file (sanity check + photo check + corresponding deposition validation)

Validate key_images, which is in the tomogram_metadata.json (and so just added to test_tomogram.py), as well as some tomogram_metadata simply sanity checks.

Also forgot to add annotation metadata sanity checks, so added that along with the annotation deposition validations.