search

DataDog / dd-trace-py

Datadog Python APM Client
https://ddtrace.readthedocs.io/
Other
532 stars 408 forks source link

ddtrace failure if pytest `no:legacypath` is set #8533

Open braingram opened 6 months ago

braingram commented 6 months ago

Summary of problem

Usage of legacy pytest path features in ddtrace prevents disabling pytest legacypath (as recommended in the pytest docs).

Running a test suite with ddtrace and -p no:legacypath results in the following error:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/_pytest/main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/_pytest/main.py", line 348, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/pluggy/_callers.py", line 34, in _multicall
INTERNALERROR>     next(gen)  # first yield
INTERNALERROR>   File "/Users/bgraham/.pyenv/versions/3.10.6/envs/romancal/lib/python3.10/site-packages/ddtrace/contrib/pytest/plugin.py", line 766, in pytest_runtest_protocol
INTERNALERROR>     root_directory = str(item.config.rootdir)
INTERNALERROR> AttributeError: 'Config' object has no attribute 'rootdir'

Which version of dd-trace-py are you using?

latest released version (2.6.5) and current github main

Which version of pip are you using?

22.2.1

Which libraries and their versions are you using?

`pip freeze` alabaster==0.7.13 appnope==0.1.3 asdf==3.1.0 asdf-astropy==0.5.0 asdf-coordinates-schemas==0.2.0 asdf-standard==1.0.3 asdf-transform-schemas==0.3.0 asdf-unit-schemas==0.1.0 asdf-wcs-schemas==0.3.0 astropy==5.3 astropy-sphinx-theme==1.1 astroquery==0.4.6 asttokens==2.2.1 attrs==23.1.0 Babel==2.12.1 backcall==0.2.0 beautifulsoup4==4.12.3 bytecode==0.15.1 cachetools==5.3.1 cattrs==23.2.3 certifi==2023.5.7 cfgv==3.4.0 chardet==5.1.0 charset-normalizer==3.1.0 ci-watson==0.6.1 colorama==0.4.6 contourpy==1.1.0 coverage==7.2.7 crds==11.17.0 cycler==0.11.0 ddsketch==2.0.4 ddtrace @ git+https://github.com/DataDog/dd-trace-py@81511c9557d46ddd5cd061d6202031610b707694 decorator==5.1.1 deepdiff==6.5.0 Deprecated==1.2.14 distlib==0.3.6 docutils==0.18.1 drizzle==1.14.3 envier==0.5.1 et-xmlfile==1.1.0 exceptiongroup==1.1.1 executing==1.2.0 filelock==3.12.2 fonttools==4.40.0 gwcs==0.20.0 html5lib==1.1 hypothesis==6.79.1 identify==2.5.29 idna==3.4 imagesize==1.4.1 importlib-metadata==6.7.0 iniconfig==2.0.0 ipython==8.14.0 jaraco.classes==3.3.1 jedi==0.19.0 Jinja2==3.1.2 jmespath==1.0.1 jsonschema==4.17.3 keyring==24.3.0 kiwisolver==1.4.4 lxml==4.9.3 MarkupSafe==2.1.3 matplotlib==3.7.1 matplotlib-inline==0.1.6 metrics-logger==0.1.0 mistune==3.0.1 more-itertools==10.2.0 nodeenv==1.8.0 numpy==1.25.0 numpydoc==1.5.0 opencv-python-headless==4.7.0.72 openpyxl==3.1.2 opentelemetry-api==1.23.0 ordered-set==4.1.0 packaging==23.1 Parsley==1.3 parso==0.8.3 pexpect==4.8.0 photutils==1.11.0 pickleshare==0.7.5 Pillow==10.0.0 platformdirs==3.8.0 pluggy==1.0.0 poppy==1.1.1 pre-commit==3.4.0 prompt-toolkit==3.0.39 protobuf==4.25.3 psutil==5.9.5 ptyprocess==0.7.0 pure-eval==0.2.2 pyerfa==2.0.0.3 Pygments==2.15.1 pyparsing==3.1.0 pyproject-api==1.5.2 pyrsistent==0.19.3 pysiaf==0.20.0 pytest==7.3.2 pytest-arraydiff==0.5.0 pytest-astropy==0.10.0 pytest-astropy-header==0.2.2 pytest-cov==4.1.0 pytest-doctestplus==0.13.0 pytest-filter-subpackage==0.1.2 pytest-mock==3.11.1 pytest-openfiles==0.5.0 pytest-remotedata==0.4.0 python-dateutil==2.8.2 pyvo==1.5 PyYAML==6.0 rad @ git+https://github.com/spacetelescope/rad.git@f6bfbfdfb0d4b6a1943cc109ada026049148ba7b requests==2.31.0 roman_datamodels @ git+https://github.com/spacetelescope/roman_datamodels.git@8dcb8dc13bdeb129fa69f4a945cbe29615bf9cfd -e git+ssh://git@github.com/braingram/romancal@1cf386593a500367bbd059cbb17c3ee812c49618#egg=romancal scipy==1.11.2 semantic-version==2.10.0 six==1.16.0 snowballstemmer==2.2.0 soc-roman-tools==0.1.0 sortedcontainers==2.4.0 soupsieve==2.5 spherical-geometry==1.2.23 Sphinx==6.2.1 sphinx-asdf==0.2.0 sphinx-astropy==1.9.1 sphinx-automodapi==0.16.0 sphinx-bootstrap-theme==0.8.1 sphinx-gallery==0.13.0 sphinx-rtd-theme==1.2.2 sphinxcontrib-applehelp==1.0.4 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-htmlhelp==2.0.1 sphinxcontrib-jquery==4.1 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 sqlparse==0.4.4 stack-data==0.6.2 stcal==1.6.0 stpipe==0.5.0 stsci-rtd-theme==1.0.0 stsci.imagestats==1.6.3 stsci.stimage==0.2.6 synphot==1.2.1 toml==0.10.2 tomli==2.0.1 tox==4.6.3 traitlets==5.9.0 tweakwcs==0.8.6 typing_extensions==4.10.0 urllib3==2.0.3 virtualenv==20.23.1 wcwidth==0.2.6 webbpsf==1.2.1 webencodings==0.5.1 wrapt==1.16.0 xmltodict==0.13.0 zipp==3.15.0

How can we reproduce your problem?

I believe (but have not confirmed) that running any test suite with --ddtrace -p no:legacypath will produce the error.

What is the result that you get?

See error above.

What is the result that you expected?

No error due to ddtrace usage of pytest legacy path.

emmettbutler commented 6 months ago

Thanks for bringing this to our attention, @braingram. We'll look into it.

@romainkomorndatadog do you think this could be related to recent changes to the pytest integration?

romainkomorndatadog commented 6 months ago

I don't think it's related to recent changes (we haven't touch our usage of rootdir).

@braingram , from reading this issue, I'm not getting the sense that this is a new error, and that this used to work in the past, is that correct?

romainkomorndatadog commented 6 months ago

I just did a quick "bisect" (with pip, not git) and it looks like this is an issue introduced in 2.x.

Tracking this with CIVIS-9265 .

romainkomorndatadog commented 6 months ago

https://github.com/DataDog/dd-trace-py/pull/7627 is what introduced the incompatibility with -p no:legacypath , although I'm assuming using ITR would also cause an error because of the places we access config.rootdir (like https://github.com/DataDog/dd-trace-py/blob/1.20/ddtrace/contrib/pytest/plugin.py#L83 and https://github.com/DataDog/dd-trace-py/blob/1.20/ddtrace/contrib/pytest/plugin.py#L96 ).

I guess we've not had combinations of -p no:legacypath and ITR usage.

braingram commented 6 months ago

Thanks all for the responses! @romainkomorndatadog you are correct that I don't believe this is a new error (we only started seeing it when we tried to use -p no:legacypath). I can also confirm that config.rootdir usage also causes errors when this option is enabled.

jamt9000 commented 4 months ago

Looks like config.rootpath is the non-legacy version

https://docs.pytest.org/en/stable/reference/reference.html#pytest.Config.rootpath