This issue emerges when a SocrataTableMetadata instance calls its .check_warehouse_data_freshness(engine) method, specifically when calling check_warehouse_data_freshness with an input value that doesn't match the expected datetime format ("%Y-%m-%dT%H:%M:%S %z"). That function is called with the instance's .latest_data_update_datetime and .latest_metadata_update_datetime attributes, and it seems (per examination of the metadata for the example data set below) that if the data set's data hasn't ever been updated, that field is None in the metadata response.
Editing the logic in check_warehouse_data_freshness() can handle this new understanding. I should also probably open an issue to refactor pytesting into an airflow container rather than in a separate (rather heavyweight) python container.
This issue emerges when a
SocrataTableMetadatainstance calls its.check_warehouse_data_freshness(engine)method, specifically when callingcheck_warehouse_data_freshnesswith an input value that doesn't match the expected datetime format ("%Y-%m-%dT%H:%M:%S %z"). That function is called with the instance's.latest_data_update_datetimeand.latest_metadata_update_datetimeattributes, and it seems (per examination of the metadata for the example data set below) that if the data set's data hasn't ever been updated, that field isNonein the metadata response.Editing the logic in
check_warehouse_data_freshness()can handle this new understanding. I should also probably open an issue to refactorpytesting into an airflow container rather than in a separate (rather heavyweight) python container.Example data set: Chicago city boundary