chanzuckerberg / cryoet-data-portal

CryoET Data Portal
MIT License
19 stars 10 forks source link

Add default source to ingestion components #1337

Open manasaV3 opened 1 week ago

manasaV3 commented 1 week ago

Currently, we have the option to add a default config if no config exists for that entity in the ingestion config. https://github.com/chanzuckerberg/cryoet-data-portal-backend/blob/2bad245f8ca41f5950de74a35401252ef0592830/ingestion_tools/scripts/importers/base_importer.py#L129

For some entities like the key photos, and alignments, there should be a fallback process when there is source specified in the ingestion config, but doesn't exist in the source bucket exists.