search

ukoethe / vigra

a generic C++ library for image analysis
http://ukoethe.github.io/vigra/
Other
409 stars 191 forks source link

How to deal with random failures in testing? #455

Closed emmenlau closed 2 years ago

emmenlau commented 5 years ago

It seems that with low probability I get failed tests in Azure. Out of 2-3 builds, roughly one fails on one of the platforms. In the past ~30 builds, I had 5-6 failed tests on Linux and 5-6 failed tests on MacOSX. Typically its the classifier test that fails, but other tests have failed as well, After a rebuild, the problems are gone. No tests have failed for me locally.

ukoethe commented 5 years ago

I noticed these failures as well (they have been in the code for a long time), but have no idea how to track the problem down, given how difficult it is to reproduce the problem locally or to debug a CI build directly. If you find a local setup that reliably reproduces the issue, I might be able to look into it. For the time being, simply restart the failing build.

emmenlau commented 2 years ago

I guess this can be closed now.