NASA-IMPACT / HLS-Transfer-Catalog

Data Catalog to manage HLS S2 transfers from ESA to NASA
Apache License 2.0
2 stars 1 forks source link

Storage ids in catalog entity #11

Closed DImuthuUpe closed 2 years ago

DImuthuUpe commented 2 years ago

There might be different source containers and destination buckets for multiple datasets. So we need two additional columns with source_storage_id and dest_storage_id. These values should be passed in the csv bulk upload as well

ub0005 commented 2 years ago

@DImuthuUpe what needs to be done if the value is empty for source_storage_id and dest_storage_id in the incoming CSV file?

Do we need to throw an error or input the value as empty string the database ?

cc: @NISH1001

DImuthuUpe commented 2 years ago

@ub0005 This should throw an error. I think it is better to enforce these rules at the database schema level as not-null fields. Then you do not need to handle them at the API manually

ub0005 commented 2 years ago

@DImuthuUpe yeah we will also enforce a not null condition on DB schema