search

ome / omero-py

Python project containing Ice remoting code for OMERO
https://www.openmicroscopy.org/omero
GNU General Public License v2.0
22 stars 32 forks source link

Retry flakey tests up to 5 times #144

Closed manics closed 4 years ago

manics commented 4 years ago

https://github.com/pytest-dev/pytest-rerunfailures Since this doesn't use test markers it's an optional dependency (omit --reruns 5)

Example output after adding this to testStringCol:

        import random
        if random.random() < 0.8:
            assert False
$ pytest -vs --reruns 5 test/unit/tablestest/test_hdfstorage.py
...
test/unit/tablestest/test_hdfstorage.py::TestHdfStorage::testStringCol RERUN
test/unit/tablestest/test_hdfstorage.py::TestHdfStorage::testStringCol RERUN
test/unit/tablestest/test_hdfstorage.py::TestHdfStorage::testStringCol RERUN
test/unit/tablestest/test_hdfstorage.py::TestHdfStorage::testStringCol RERUN
test/unit/tablestest/test_hdfstorage.py::TestHdfStorage::testStringCol PASSED
...
================ 14 passed, 2 xfailed, 4 rerun in 1.82 seconds =================
joshmoore commented 4 years ago

Travis is green (I didn't see a re-run but the "sugar" logging may have hidden it) Glad to get this in to deal with the current PyTables flakiness. The only caveat though is if that we may start seeing similar flakiness in production with concurrent OMERO.tables access. Hopefully https://github.com/ome/omero-py/issues/58#issuecomment-561669494 etc. will remind us to test.