A common pattern I see when classifying failures is that I want to do two matches, on separate parts of the log. It's something like:
I want to test that a particular test failed (e.g., TestCaffe2End2End::test_alexnet FAILED)
I want to test that this test failed in a particular way (e.g., python/models/download.py:84: BadStatusLine)
I don't want to test for these individually: if I match for (1) I will mark all such failures as flaky, even if the contents of test failure could be different. If I match for (2) I may match for too many cases.
A common pattern I see when classifying failures is that I want to do two matches, on separate parts of the log. It's something like:
TestCaffe2End2End::test_alexnet FAILED
)python/models/download.py:84: BadStatusLine
)I don't want to test for these individually: if I match for (1) I will mark all such failures as flaky, even if the contents of test failure could be different. If I match for (2) I may match for too many cases.