search

smarie / python-pytest-harvest

Store data created during your `pytest` tests execution, and retrieve it at the end of the session, e.g. for applicative benchmarking purposes.
https://smarie.github.io/python-pytest-harvest/
BSD 3-Clause "New" or "Revised" License
61 stars 8 forks source link

tests fail with fixture 'fixture_store' not found #24

Closed keszybz closed 4 years ago

keszybz commented 4 years ago

When I run python3 -m pytest -v from a git checkout, many tests fail with the same error:

___________________________________________ ERROR at setup of test_my_app_bench[B-1] ____________________________________________
file /home/zbyszek/python/python-pytest-harvest/pytest_harvest/tests/test_doc_example.py, line 23
  @pytest.mark.parametrize("algo_param", [1, 2], ids=str)
  def test_my_app_bench(algo_param, dataset, results_bag):
file <makefun-gen-4>, line 1: source code not available
E       fixture 'fixture_store' not found
>       available fixtures: LineMatcher, _config_for_test, _pytest, _sys_snapshot, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, celery_app, celery_config, celery_enable_logging, celery_includes, celery_parameters, celery_session_app, celery_session_worker, celery_worker, celery_worker_parameters, celery_worker_pool, cov, dataset, depends_on_current_app, doctest_namespace, linecomp, monkeypatch, no_cover, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, testdir, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory, use_celery_app_trap
>       use 'pytest --fixtures [testpath]' for help on them.

If I do pip3 install --user ., many of those tests start passing. It would be great if the tests could run before installation.

smarie commented 4 years ago

This is an issue that is common to all pytest plugins I think: the plugin is only registered with pytest if it the package is installed. That is when its entry points are registered and the special pytest entry point is detected by pytest.

So there is not much I can do here. What did you try to do ?

EDIT: for development, I typically do pip install -e . so that the package is installed in debug mode and I can continue to work on it without re-installing everytime I want to test.

keszybz commented 4 years ago

Ideally, the test suite would be structured so that it is possible to run tests before installation, because it makes development and packaging easier... Dunno, maybe it should tweak some pytest config internally. But if this can't be solved, workarounds are easy enough.

(I'm trying to make a package for Fedora, and PYTHONPATH=%{buildroot}/%{python3_sitelib} %{__python3} -m pytest -v works well enough.)

smarie commented 4 years ago

I agree with the "ideally" but that's how pytest seems to be working :)

Happy to see that the workarounds are working for you. I'll close the issue now, please do not hesitate to reopen if needed.