Closed hoxbro closed 1 month ago
Attention: Patch coverage is 97.91667%
with 1 lines
in your changes are missing coverage. Please review.
Project coverage is 88.47%. Comparing base (
945e179
) to head (26f7d46
). Report is 5 commits behind head on main.
Files | Patch % | Lines |
---|---|---|
holoviews/tests/ipython/test_magics.py | 85.71% | 1 Missing :warning: |
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Locally, I have run the following script which uses pytest-random-order
on https://github.com/holoviz/holoviews/pull/6234/commits/f0541c6cd2cbae2c65166003f1c2b12bdda5d4f9
mkdir -p logs
for i in {01..100}; do
echo "Run $i"
pytest holoviews/tests -n logical --dist loadgroup --random-order --color=no > logs/log$i.txt
done
This resulted in the following:
❯ rg FAILED logs | sort
logs/log12.txt:FAILED holoviews/tests/ipython/test_magics.py::TestOptsMagic::test_cell_opts_plot_dynamic
logs/log28.txt:FAILED holoviews/tests/plotting/bokeh/test_callbacks.py::TestCallbacks::test_callback_cleanup
logs/log56.txt:FAILED holoviews/tests/ipython/test_magics.py::TestOptsMagic::test_cell_opts_plot_dynamic
logs/log57.txt:FAILED holoviews/tests/plotting/bokeh/test_callbacks.py::TestCallbacks::test_callback_cleanup
logs/log61.txt:FAILED holoviews/tests/plotting/bokeh/test_callbacks.py::TestCallbacks::test_callback_cleanup
logs/log72.txt:FAILED holoviews/tests/plotting/bokeh/test_server.py::TestBokehServer::test_server_dynamicmap_with_stream
logs/log78.txt:FAILED holoviews/tests/plotting/bokeh/test_server.py::TestBokehServer::test_server_dynamicmap_with_dims
logs/log81.txt:FAILED holoviews/tests/plotting/bokeh/test_server.py::TestBokehServer::test_server_dynamicmap_with_dims
logs/log94.txt:FAILED holoviews/tests/ipython/test_magics.py::TestOptsMagic::test_cell_opts_style_dynamic
❯ rg FAILED logs --json | jq '[select(.type == "match")][].data.path.text' | xargs rg seed | sort
logs/log12.txt:Using --random-order-seed=94455
logs/log28.txt:Using --random-order-seed=484446
logs/log56.txt:Using --random-order-seed=735748
logs/log57.txt:Using --random-order-seed=876950
logs/log61.txt:Using --random-order-seed=72331
logs/log72.txt:Using --random-order-seed=438329
logs/log78.txt:Using --random-order-seed=47376
logs/log81.txt:Using --random-order-seed=747979
logs/log94.txt:Using --random-order-seed=479472
Running with those seeds again, the test did not fail, so I have marked them as flaky.
Isolated test failures (from https://github.com/holoviz/holoviews/issues/5565)
Status | Command |
---|---|
:green_circle: | pytest holoviews/tests -k InspectorTests |
:red_circle: * | pytest holoviews/tests/core/data/test_imageinterface.py::RGBElement_ImageInterfaceTests::test_reduce_to_single_values |
:green_circle: | holoviews/tests/core/test_options.py::TestCrossBackendOptionSpecification::test_mpl_bokeh_mpl_via_builders_opts_method |
*) Though, it is now consistently failing...
Fixes #5565
This time it has been in a much better state than when I looked over a year ago. I think this relates to work done in #5954 and #6200, where I made it possible to run tests with required dependencies + pytest.
The main change in this PR is to reset
hv.Store
before each run.