Open kltm opened 5 years ago
Note that we already have: https://github.com/geneontology/go-site/blob/master/scripts/sanity-check-ann-report.py This failed to find the issue as it is looking for something rather more severe, at the ~50% mark, rather than the ~8% mark that this would have shown up at. The gross number is at least partially due to the relatively large effects on smaller files. However, this ticket is wanting to have what is essentially an ex post facto check -- we know that there was 424986 lines last time around, why the 5% reduction? This is, naturally, much more brittle and would require regular updates of the metadata file; that said, I think a lot less unseen changes will sneak through this way.
Would start with something like:
- "filename": rgd.gaf
"expected-lines": 424986
"acceptable-variation": 0.05
This might actually be a nice place to get behave in place again. Then again, YAML is nice and easy and ubiquitous for us at this point.
Noting that this will branch into the JSON outputs from @lpalbou as well when defined.
@pgaudet #1152 Is a good place to add these before mainlining them.
This needs to run on snapshot.
From meeting with @pgaudet and @lpalbou
Examples:
We are too frequently bitten by issues like https://github.com/geneontology/go-site/issues/1061 , where there is an unseen drop in annotations that turns out to presage an error somewhere in the pipeline.
We need a sanity check, either as separate metadata or in the main metadata, that puts a range on expected annotation volume and either crashes the pipeline or emails.