w3c / strudy

Strudy is an analysis tool for Web spec crawl reports created by Reffy
MIT License
5 stars 5 forks source link

Run IDL analyses on curated branch, enable noEvent #787

Closed tidoust closed 2 months ago

tidoust commented 2 months ago

This duplicates the calls to Strudy in the analysis job: first pass analyzes the raw crawl results, second pass runs further analyses (not the same ones!) on the curated crawl results. IDL analyses are now done as part of the second pass.

The rationale is that some IDL anomalies may be indirectly triggered by another IDL hiccup that we already identified and reported while doing data curation in Webref. This makes it at least theoretically possible to end up in a situation where we detect an anomaly twice (from two different angles). It seems better to run further IDL analyses on the curated branch instead.

This is particularly needed for the new noEvent anomaly, which would otherwise report problems that our events patching already takes care of.

The noEvent anomaly will create two issues next time the job runs: