w3c / reffy

Reffy is a Web spec crawler and analyzer tool. It is notably used to update Webref
MIT License
71 stars 23 forks source link

Force full crawl on spec when previous crawl reported an error #1602

Closed tidoust closed 4 months ago

tidoust commented 4 months ago

Reffy happily reused the result of a previous crawl on a spec when a spec wasn't modified even though these results indicated an error.

In most cases, the error is a network error, and reusing the result is actually quite fine because that result was for when the spec could be crawled without error, and we should just forget about the error.

In some cases though, the error is for something else, and reusing the result while pretending the error is gone would be wrong.

These cases are not straightforward to distinguish. This update does not try to be smart and simply forces a full crawl on specs for which the previous result contains an error.