Adds feature to filter newly ingested records with row-wise hash values.
This provides the benefits outlined in #123.
Removes the dependence on prop_ids
Forces reprocessing on any dataset changes
This improves the previous workflow (implemented last week) to filter by prop_id which combined different parts of the workflow which should remain disconnected.
Also - this workflow functions on any dataset (including OXCGRT) which is also an improvement.
Adds feature to filter newly ingested records with row-wise hash values.
This provides the benefits outlined in #123.
This improves the previous workflow (implemented last week) to filter by
prop_idwhich combined different parts of the workflow which should remain disconnected.Also - this workflow functions on any dataset (including OXCGRT) which is also an improvement.
Closes #123