search

WithSecureOpenSource / pytest-rts

Coverage-based regression test selection (RTS) plugin for pytest
Apache License 2.0
7 stars 3 forks source link

problems running pytest-rts first time #78

Closed matveypashkovskiy closed 3 years ago

matveypashkovskiy commented 3 years ago

Created a small project to reproduce coverage percentage problem but run into problems during the mapping database initialization.

If I run make test-rts it fails with:

=========================================================================== test session starts ============================================================================
platform darwin -- Python 3.6.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /Users/pashma/workspace/pytest-rts-cov
plugins: cov-2.11.1, rts-1.1.10
collected 1 item                                                                                                                                                           
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/_pytest/main.py", line 333, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/_pytest/main.py", line 638, in perform_collect
INTERNALERROR>     session=self, config=self.config, items=items
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pytest_rts/pytest/init_phase_plugin.py", line 36, in pytest_collection_modifyitems
INTERNALERROR>     for testfile_path in self.testfiles
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/pytest_rts/pytest/init_phase_plugin.py", line 36, in <dictcomp>
INTERNALERROR>     for testfile_path in self.testfiles
INTERNALERROR>   File "/Users/pashma/workspace/pytest-rts-cov/.venv/lib/python3.6/site-packages/coverage/python.py", line 61, in get_python_source
INTERNALERROR>     raise NoSource(exc_msg)
INTERNALERROR> coverage.misc.NoSource: No source for code: 'tests/foo_test.py'.
INTERNALERROR> Aborting report output, consider using -i.

========================================================================== no tests ran in 0.06s ===========================================================================
make: *** [test-rts] Error 3

It looks like a config problem of coverage.py.

guotin commented 3 years ago

Looking at the trace, the tool calculates the test file path from the single pytest item it finds and tries to read the source code for it with Coverage.py to define where the test function starts and ends. coverage.misc.NoSource is thrown if it can't find the file for some reason. I can't get this example project to fail. I did the following:

  1. cd pytest_rts_cov/
  2. git init
  3. git add .
  4. git commit -m "initial"
  5. python3 -m venv .venv
  6. source .venv/bin/activate
  7. pip install -r requirements.txt
  8. make test-rts

And got a succesful run:

. .venv/bin/activate; pytest -x \
                    --rts \
                    --cov-config=.coveragerc \
                    --cov=pytest_rts_cov \
                    --cov-report=html \
                    --cov-report=term \
#                       --cov-fail-under=89.8
No mapping database detected, starting initialization...
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.5, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: /home/eero/Desktop/pytest_rts_cov
plugins: cov-2.11.1, rts-1.1.10
collected 1 item                                                                                                                                                                                                  

pytest_rts_cov/tests/foo_test.py .                                                                                                                                                                          [100%]

----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name                         Stmts   Miss  Cover
------------------------------------------------
pytest_rts_cov/__init__.py       0      0   100%
pytest_rts_cov/bar.py            3      3     0%
pytest_rts_cov/foo.py            3      1    67%
------------------------------------------------
TOTAL                            6      4    33%
Coverage HTML written to dir htmlcov
matveypashkovskiy commented 3 years ago

Hm, could you try with Python 3.6?

matveypashkovskiy commented 3 years ago

Ok, looks like the problem is in type hints. When I removed them from the code it start to work with both versions.