Open shonfeder opened 3 weeks ago
Related to the context where I encountered this issue, I'm relatively biased here. This is definitely a great idea, and it would save time to maintainers. About the first proposition, the heuristics need to be added carefully to prevent broken releases to make their way to opam. It is easier to ignore CI errors than not having them reported.
When new package releases encounter revdeps errors, it can be really time consuming (and probably pretty unreliable) to try to audit the failures to see if the build/test failures have any relationship to the new release. We can use this issue to gather ideas on how to make the reported results more reliable and easier to navigate.
Two ideas to try to improve the relevance of revdeps errors:
Both of these could benefit from reviewer input during review time, allowing reviewers to mark revdeps failures as irrelevant, e.g., with the addition of a regex on certain errors, or by marking problem packages.
Another improvement that is already planned is to not report errors for packages that fail due to an intermediate dependency failure: e.g., if we have a dependency chain like
A <- B <- C
andB
fails, we do not also want a report on the inevitable failure ofC
due toB
.