tl-its-umich-edu / unizin-validation

Unizin Validation Scripts
1 stars 4 forks source link

Compare results of run to previous day and fail if there's a large discrepancy #28

Open jonespm opened 3 years ago

jonespm commented 3 years ago

A few times we've had an issue come up where the results from the days run with 2x the previous days run. We should either

We should set a difference indicator, I feel like it could be different per table and whether it's an increase or decrease, but +- 25% seems like a good start?

jonespm commented 3 years ago

The most common time we see this problem is when a table has double the results from the previous run. (Seen in the past in submission_dim)

jonespm commented 3 years ago

There was an issue today where course_dim had 0.5% less courses than the day before. This was mentioned as a case that would be worth notifying on. If something drops by a specified amount. But we'll need to store this data.