brighthive / data-resource-api

An elegant, opinionated framework for deploying BrightHive Data Resources with zero coding.
MIT License
4 stars 1 forks source link

Data Resource Manager crashes when non-JSON file is encountered #44

Open gregmundy opened 4 years ago

gregmundy commented 4 years ago

If a non-JSON file is encountered in the location where the data resource API retrieves files it will crash. This sometimes happens when configurations are pulled from a repository that may have additional files (such as README). The Data Resource Manager should gracefully ignore this invalid file and continue to process valid files. Ideally, this issue should be seen in logs.

loganripplinger commented 4 years ago

Probably just need to wrap the json.loads() commands in try/except and catch JSONDecodeError per https://docs.python.org/3/library/json.html

https://docs.python.org/3/library/json.html#json.JSONDecodeError

https://github.com/brighthive/data-resource-api/blob/8c30dc334495bf8d84959c337f21a89c604828fb/data_resource_api/app/data_resource_manager.py#L228

https://github.com/brighthive/data-resource-api/blob/8c30dc334495bf8d84959c337f21a89c604828fb/data_resource_api/app/data_model_manager.py#L307