Flaky Tests - Githubissues

Epistimio / orion

Asynchronous Distributed Hyperparameter Optimization.

https://orion.readthedocs.io

Other

287 stars 52 forks source link

Flaky Tests #1076

Open Delaunay opened 1 year ago

Delaunay commented 1 year ago

Recently, tests having requiring a lot of re-runs. Tests are too long for reruns to be acceptable.

[Parallel] FAILED tests/unittests/core/database/test_pickleddb.py::TestConcurreny::test_concurrent_unique_writes - Failed: Timeout >360.0s
FAILED tests/functional/algos/test_algos.py::test_parallel_workers[pb2] - Failed: Timeout >360.0s
FAILED tests/functional/algos/test_algos.py::test_branching_algos[pb2] - Failed: Timeout >360.0s
The dash tests

Solved

FAILED tests/unittests/client/test_runner.py::test_runner_inside_dask - AssertionError in https://github.com/Epistimio/orion/pull/1072

Delaunay commented 1 year ago

https://docs.pytest.org/en/7.1.x/explanation/flaky.html#xfail-strict