Where the relevant condition is whether the data freshness check indicates there's fresher data (fresher than what's already in the data database) to be retrieved from the data source.
If data is successfully pulled:
new or preexisting-but-updated records should be ingested to the correct table,
the record for that freshness check should be updated to indicate data was pulled.
If our data warehouse already has fresh data, do nothing.
MattTriano
commented
1 year ago
Where the relevant condition is whether the data freshness check indicates there's fresher data (fresher than what's already in the data database) to be retrieved from the data source.
If data is successfully pulled:
If our data warehouse already has fresh data, do nothing.