Closed agstephens closed 3 years ago
The ideal solution to this would be:
NEED TO TEST THIS AGAINST A REAL EXAMPLE TO PROVE RESULT IS THE SAME AFTER PUTTING IN THIS FILTER
This way, we only end up with a few bad records being excluded, rather than lots.
Simon fixed this problem. Just need to rerun over failures. No action required in our ingestion code.
I'm processing the land cdmlite data now. It is generally going well but there are some observations that don't find a valid match to a "source_id" in either the sub-daily or the daily+monthly Source Configuration files you provided.
Here is an example:
primary_id = CA001091174 record_number = 2 frequency = daily
How do you think we should handle these?