codecov / feedback

A place to discuss feedback about the pull request and web product experience.
37 stars 7 forks source link

[Feature Idea] Flaky coverage detection. #561

Closed drazisil-codecov closed 3 weeks ago

drazisil-codecov commented 3 weeks ago

However, looking at it further, I'm increasingly convinced that this is caused by something other than flags, maybe just code coverage measurement being a little flaky and varying by a line of two from one build to another for no good reason. I should really set up some thresholds for red notifications to avoid this at some point...

Originally posted by @HadrienG2 in https://github.com/codecov/feedback/discussions/560#discussioncomment-11095869

We have flaky test detection, how hard would it be to create flaky coverage detection? It would be great if we could identity and report if we see lines or files jumping from covered to uncovered and back.