Add sanity check that we are regularly adding new versions

Mr0grog commented 7 years ago

While we can try and write our importing tools to throw and notify about errors as much as possible, there can always be silent failures we miss, e.g. https://github.com/edgi-govdata-archiving/web-monitoring-versionista-scraper/issues/51

It might be a useful sanity check to regularly make sure new versions are being added to the DB. Some approaches:

Have the DB run scheduled job that regularly checks whether there have been no new versions for some period of time (a few hours, maybe?)
Have the DB alert if there was an import with no versions? (Probably not good enough since this would miss a silent failure that causes an import script to stop before getting to the import step.)
Add a way for the importer scripts to alert if there were no versions found or to keep track of previous runs and alert if nothing was found across several.

I’m thinking (1) is probably the best bet here since it is least hobbled by anything that could happen in an import script and because it can happen in one place, but flagging this here rather than on the DB project in case there are other opinions.

Any thoughts, @danielballan?

danielballan commented 6 years ago

The single best human-readable metric I can think of is a histogram of Versions per Page added in a given day.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in seven days if no further activity occurs. If it should not be closed, please comment! Thank you for your contributions.

Mr0grog commented 5 years ago

Still relevant, and something we should do alongside @danielballan’s upcoming metrics plans!

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in seven days if no further activity occurs. If it should not be closed, please comment! Thank you for your contributions.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in seven days if no further activity occurs. If it should not be closed, please comment! Thank you for your contributions.

edgi-govdata-archiving / web-monitoring

Add sanity check that we are regularly adding new versions #80