Remediate Missing Versionista Data Kendra Found

[Mr0grog]

Kendra notes:

https://monitoring.envirodatagov.org/page/7cbff3dc-85a5-4a54-b005-a88798adc465/acc55f72-508b-4d6e-b0f1-a7f9c99bc88e..60fc82ca-2d98-4105-8ac8-073a033d3428

Did Versionista do better, same, or worse? How so? I was interested to see that the last version saved by Versionista is more recent than that of the EDGI software, and indicates that the System Notification was removed on August 19th, just two days after it was added - thus, Versionista appears to give a better idea of when this change took place than the EDGI software does.

The missing version is from Aug 20, 2017 2:21 AM UTC. We have no records of this version at all in raw scraper backups, import records or other historical archives, either, so it’s likely there are other versions in a similar time band that are missing. We somehow never saw this version in the first place.

Need to:

• [x] Do a full scan of all versions in our DB and generate a mapping of versionista version → Scanner UUID
• [x] Do a full scan (as lightweight as possible) of all versions still in Versionista (in progress; takes ~9 hours per account)
• [x] Reconcile the two to find what Versionista versions that are still retrievable are missing
• [x] Retrieve and import those versions

[Mr0grog]

FINALLY done. Ultimately collected 5,284 missing versions across both accounts (or ~1.1% of Versionista versions).

[danielballan]

Good save.

[lightandluck]

🙏🏽

[weatherpattern]

many thanks for catching this.