Open kloczek opened 5 months ago
The code in exercises.py
is run by pytest-perf
. pytest-perf
requires a git repo because it compares the performance of the code under test against the main branch. Since you're running against a copy of the source without a remote, the perf tests will not run, or even if they did run, they won't provide any useful information.
I wonder if pytest-perf should just not run when there's no git remote available. Or maybe integrators should disable the plugin by running with -p no:perf
or by not installing the pytest-perf
plugin.
Is there a reason this issue didn't occur previously?
The code in exercises.py is run by pytest-perf. pytest-perf requires a git repo because it compares the performance of the code under test against the main branch.
So it is kind of benchmark test and not kind of functional one. Am I right? Maybe it would be good to fire that unit only when .git/ directory exist? 🤔
It is something of a benchmark. It's a comparative benchmark. And currently, the test can never fail. It's there mainly to emit the differences in performance during a PR or to track performance changes over time. For example, if a user were to commit a change that reduced the performance noticeably, we could go back to the CI and see when that performance regression occurred.
Maybe it would be good to fire that unit only when .git/ directory exist? 🤔
Yes, I'm considering that. But I'm also interested in knowing the answer to my question.
Is there a reason this issue didn't occur previously?
i.e. Why is this issue reported against 7.2.0 even though the performance tests have been there for many versions?
Just FYI, openSUSE runs the tests with
%pytest --ignore exercises.py
That’s, I guess, one way how to overcome this, right?
I'm not is it intentional but pytest executed with source tree settings is scanning exercises.py and it fails in units found int that file
Here is summary pytest output:
```console ----------------------------- Captured stderr call ----------------------------- fatal: not a git repository (or any parent up to mount point /home/tkloczko) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). ===================================== perf ===================================== exercises.py:cached distribution: None exercises.py:discovery: None exercises.py:entry_points(): None exercises.py:entrypoint_regexp_perf: None exercises.py:uncached distribution: None =========================== short test summary info ============================ SKIPPED [1] tests/compat/test_py39_compat.py:56: Tests specific for Python 3.8/3.9 FAILED exercises.py::exercises.py:cached distribution - subprocess.CalledProc... FAILED exercises.py::exercises.py:discovery - subprocess.CalledProcessError: ... FAILED exercises.py::exercises.py:entry_points() - subprocess.CalledProcessEr... FAILED exercises.py::exercises.py:entrypoint_regexp_perf - subprocess.CalledP... FAILED exercises.py::exercises.py:uncached distribution - subprocess.CalledPr... =================== 5 failed, 106 passed, 1 skipped in 1.88s =================== ```