Mr0grog
commented
3 years ago I’m going to go ahead and merge this without a review once tests pass since it’s a hotfix, and it looks like these have been failing for a few days.
Closed Mr0grog closed 3 years ago
Mr0grog
commented
3 years ago I’m going to go ahead and merge this without a review once tests pass since it’s a hotfix, and it looks like these have been failing for a few days.
This fixes issues that have cropped up in two counties (Santa Clara and Contra Costa) that are breaking their scrapers:
In Contra Costa, the specifics of the issue I worked around in #217 have changed slightly -- now the first day with data is prefixed by a month of records with case counts of 0. It seems like these are probably fake, since the first day with data still exhibits the same problem that the first day used to, that its total is clearly accounting for some earlier records that are not shown in the public data we can see. The fix is basically the same, except it's no longer about the first day; it's about the first day with non-zero data.
In Santa Clara, the final row has no date. Since it's only one row and will hopefully be fixed on the county's side, I've changed the code to just skip the record and log a warning, but otherwise operate fine. If the warning would have applied to every record, though, we know something else is wrong and it raises a fatal error.