Closed waterson closed 2 weeks ago
When I look at the output logs for matplotlib__matplotib-18869, I see the following:
+ pytest --no-header -rA --tb=no -p no:cacheprovider lib/matplotlib/tests/test_matplotlib.py ============================= test session starts ============================== collected 8 items lib/matplotlib/tests/test_matplotlib.py ....s... [100%] =========================== short test summary info ============================ PASSED lib/matplotlib/tests/test_matplotlib.py::test_parse_to_version_info[3.5.0-version_tuple0] PASSED lib/matplotlib/tests/test_matplotlib.py::test_parse_to_version_info[3.5.0rc2-version_tuple1] PASSED lib/matplotlib/tests/test_matplotlib.py::test_parse_to_version_info[3.5.0.dev820+g6768ef8c4c-version_tuple2] PASSED lib/matplotlib/tests/test_matplotlib.py::test_parse_to_version_info[3.5.0.post820+g6768ef8c4c-version_tuple3] PASSED lib/matplotlib/tests/test_matplotlib.py::test_importable_with_no_home PASSED lib/matplotlib/tests/test_matplotlib.py::test_use_doc_standard_backends PASSED lib/matplotlib/tests/test_matplotlib.py::test_importable_with__OO SKIPPED [1] lib/matplotlib/tests/test_matplotlib.py:20: chmod() doesn't work as root ========================= 7 passed, 1 skipped in 9.47s =========================
But this instance fails to pass because of test_tmpconfigdir_warning.
test_tmpconfigdir_warning
"PASS_TO_PASS": { "success": [ "lib/matplotlib/tests/test_matplotlib.py::test_importable_with_no_home", "lib/matplotlib/tests/test_matplotlib.py::test_use_doc_standard_backends", "lib/matplotlib/tests/test_matplotlib.py::test_importable_with__OO" ], "failure": [ "lib/matplotlib/tests/test_matplotlib.py::test_tmpconfigdir_warning" ] },
Since the test is never run, should this be removed from the PASS_TO_PASS for this instance?
Run evaluation for `matplotlib__matplotlib-18869 using SWE-bench_Lite
Instance passes given the above output
Instance fails because test_tmpconfigdir_warning is never run
No response
Oops... running with an older version here. Sorry!
Describe the bug
When I look at the output logs for matplotlib__matplotib-18869, I see the following:
But this instance fails to pass because of
test_tmpconfigdir_warning
.Since the test is never run, should this be removed from the PASS_TO_PASS for this instance?
Steps/Code to Reproduce
Run evaluation for `matplotlib__matplotlib-18869 using SWE-bench_Lite
Expected Results
Instance passes given the above output
Actual Results
Instance fails because
test_tmpconfigdir_warning
is never runSystem Information
No response