karthiknadig commented 5 years ago

Environment data

VS Code version: 1.35.1
Extension version (available under the Extensions sidebar): 2019.6.22090
OS and version: Windows
Python version (& distribution if applicable, e.g. Anaconda): 3.7.3
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A
Relevant/affected Python packages and their versions: pytest
Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977): N/A

Expected behaviour

Tests should be discovered.

Actual behaviour

Steps to reproduce:

Clone ptvsd : https://github.com/Microsoft/ptvsd
Switch to dbg_adapter_refactor branch

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)


python C:\Users\kanadig\.vscode\extensions\ms-python.python-2019.6.22090\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear
Test Discovery failed: 
Error: I+00.047: Test environment:

          CPU count: 12

          System paths:
              sys.prefix: C:\Python37
              sys.base_prefix: C:\Python37
              sys.real_prefix: <missing>
              site.getsitepackages(): C:\Python37
                                      C:\Python37\lib\site-packages
              site.getusersitepackages(): C:\Users\kanadig\AppData\Roaming\Python\Python37\site-packages
              sys.path (site-packages): C:\Python37\lib\site-packages
                                        C:\Users\kanadig\AppData\Roaming\Python\Python37\site-packages
              sysconfig.get_path('stdlib'): C:\Python37\Lib
              sysconfig.get_path('platstdlib'): C:\Python37\Lib
              sysconfig.get_path('purelib'): C:\Python37\Lib\site-packages
              sysconfig.get_path('platlib'): C:\Python37\Lib\site-packages
              sysconfig.get_path('include'): C:\Python37\Include
              sysconfig.get_path('scripts'): C:\Python37\Scripts
              sysconfig.get_path('data'): C:\Python37
              os.__file__: C:\Python37\lib\os.py
              threading.__file__: C:\Python37\lib\threading.py

Note that pytest is able to discover tests if run directly

py -3.7 -m pytest --collect-only

```console C:\GIT\ptvsd>py -3.7 -m pytest --collect-only ================================================= test session starts ================================================= platform win32 -- Python 3.7.3, pytest-4.3.0, py-1.8.0, pluggy-0.9.0 I+00.047: Test environment: CPU count: 12 System paths: sys.prefix: C:\Python37 sys.base_prefix: C:\Python37 sys.real_prefix: site.getsitepackages(): C:\Python37 C:\Python37\lib\site-packages site.getusersitepackages(): C:\Users\kanadig\AppData\Roaming\Python\Python37\site-packages sys.path (site-packages): C:\Python37\lib\site-packages C:\Users\kanadig\AppData\Roaming\Python\Python37\site-packages sysconfig.get_path('stdlib'): C:\Python37\Lib sysconfig.get_path('platstdlib'): C:\Python37\Lib sysconfig.get_path('purelib'): C:\Python37\Lib\site-packages sysconfig.get_path('platlib'): C:\Python37\Lib\site-packages sysconfig.get_path('include'): C:\Python37\Include sysconfig.get_path('scripts'): C:\Python37\Scripts sysconfig.get_path('data'): C:\Python37 os.__file__: C:\Python37\lib\os.py threading.__file__: C:\Python37\lib\threading.py rootdir: C:\GIT\ptvsd, inifile: pytest.ini plugins: xdist-1.26.1, timeout-1.3.3, forked-1.0.2, cov-2.6.1 timeout: 30.0s timeout method: thread timeout func_only: False collected 998 items ============================================ no tests ran in 0.68 seconds ============================================= ```

brettcannon commented 5 years ago

Is this because of #6347? (E.g. can you discover if you run the pytest command instead of python -m pytest?)

aminfara commented 5 years ago

I also have "Test discovery failed" after upgraded to 2019.6.22090

VS Code version: 1.36.0 Extension version (available under the Extensions sidebar): 2019.6.22090 OS and version: Linux Python version (& distribution if applicable, e.g. Anaconda): 3.6.5 Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A Relevant/affected Python packages and their versions: pytest - 3.2.1 Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977): False

in my case I have the following trace:

python ..../.vscode-server/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear

Traceback (most recent call last):
  File ".../lib/python3.6/site-packages/_pytest/main.py", line 648, in perform_collect
    config=self.config, items=items)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "..../.vscode-server/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 76, in pytest_collection_modifyitems
    test, parents = parse_item(item, self.NORMCASE, self.PATHSEP)
  File "..../.vscode-server/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 191, in parse_item
    for marker in item.own_markers:
AttributeError: 'Function' object has no attribute 'own_markers'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../lib/python3.6/site-packages/_pytest/main.py", line 110, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File ".../lib/python3.6/site-packages/_pytest/main.py", line 145, in _main
    config.hook.pytest_collection(session=session)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File ".../lib/python3.6/site-packages/_pytest/main.py", line 155, in pytest_collection
    return session.perform_collect()
  File ".../lib/python3.6/site-packages/_pytest/main.py", line 650, in perform_collect
    hook.pytest_collection_finish(session=self)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File ".../lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "..../.vscode-server/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 90, in pytest_collection_finish
    test, parents = parse_item(item, self.NORMCASE, self.PATHSEP)
  File "..../.vscode-server/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 191, in parse_item
    for marker in item.own_markers:
AttributeError: 'Function' object has no attribute 'own_markers'

Platform:

platform linux -- Python 3.6.5, pytest-3.2.1, py-1.4.34, pluggy-0.4.0

I confirm that with Extension version 2019.5.18875, (all other things the same) it discovers my tests and run/debug pytest just fine.

karrtikr commented 5 years ago

@aminfara Your traces are different, so I've opened a separate issue for you. Please follow that.

karthiknadig commented 5 years ago

@brettcannon Yes it is able to discover if i use pytest executable.

pytest.exe --collect-only

```console ================================================== test session starts =================================================== platform win32 -- Python 3.7.2, pytest-4.3.0, py-1.8.0, pluggy-0.9.0 I+00.297: Test environment: CPU count: 4 System paths: sys.prefix: c:\python37 sys.base_prefix: c:\python37 sys.real_prefix: site.getsitepackages(): c:\python37 c:\python37\lib\site-packages site.getusersitepackages(): C:\Users\karth\AppData\Roaming\Python\Python37\site-packages sys.path (site-packages): C:\Users\karth\AppData\Roaming\Python\Python37\site-packages c:\python37\lib\site-packages sysconfig.get_path('stdlib'): c:\python37\Lib sysconfig.get_path('platstdlib'): c:\python37\Lib sysconfig.get_path('purelib'): c:\python37\Lib\site-packages sysconfig.get_path('platlib'): c:\python37\Lib\site-packages sysconfig.get_path('include'): c:\python37\Include sysconfig.get_path('scripts'): c:\python37\Scripts sysconfig.get_path('data'): c:\python37 os.__file__: c:\python37\lib\os.py threading.__file__: c:\python37\lib\threading.py rootdir: C:\GIT\ptvsd, inifile: pytest.ini plugins: xdist-1.26.1, timeout-1.3.3, forked-1.0.2, cov-2.6.1 timeout: 30.0s timeout method: thread timeout func_only: False collected 998 items ============================================== no tests ran in 1.41 seconds ============================================== ```

@karrtikr I think the issue in this particular case is that test discovery does not use the .env file.

karthiknadig commented 5 years ago

Running test discovery like this works better:

cd c:\GIT\ptvsd && cmd /C "set "PYTHONPATH=C:\GIT\ptvsd\src" && py -3.7 C:\Users\karth\.vscode\extensions\ms-python.python-2019.6.22090\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear"

DonJayamanne commented 5 years ago

Prescribed Solution

Include cwd into current path (insert instead of replace) Modify the following code in pythonFiles/testing_tools/run_adapter.py
```
sys.path[0] = os.path.dirname(
os.path.dirname(
    os.path.abspath(__file__)))
```

ericsnowcurrently commented 5 years ago

The fix to the pytest adapter did not improve the outcome. Discovery fails for me:

From "Python Test Log" Output pane:

```console python $HOME/.vscode-server/extensions/ms-python.python-2019.7.27777-dev/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear . Test Discovery failed: Error: I+00.116: ptvsd logs will be in $HOME/ptvsd/tests/_logs ============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-4.6.4, py-1.8.0, pluggy-0.12.0 I+00.127: Test environment for tests-12524: CPU count: 8 System paths: sys.prefix: /usr sys.base_prefix: /usr sys.real_prefix: site.getsitepackages(): /usr/lib/python3.7/dist-packages /usr/lib/python3/dist-packages /usr/local/lib/python3.7/dist-packages site.getusersitepackages(): $HOME/.local/lib/python3.7/site-packages sys.path (site-packages): $HOME/.local/lib/python3.7/site-packages sysconfig.get_path('stdlib'): /usr/lib/python3.7 sysconfig.get_path('platstdlib'): /usr/lib/python3.7 sysconfig.get_path('purelib'): /usr/lib/python3.7/site-packages sysconfig.get_path('platlib'): /usr/lib/python3.7/site-packages sysconfig.get_path('include'): /usr/include/python3.7m sysconfig.get_path('scripts'): /usr/bin sysconfig.get_path('data'): /usr os.__file__: /usr/lib/python3.7/os.py threading.__file__: /usr/lib/python3.7/threading.py rootdir: $HOME/ptvsd, inifile: pytest.ini plugins: asyncio-0.8.0, xdist-1.29.0, forked-1.0.2, timeout-1.3.3 timeout: 30.0s timeout method: thread timeout func_only: False collected 1240 items / 8 errors / 1232 selected ==================================== ERRORS ==================================== _ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_check_pydevconsole.py _ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_check_pydevconsole.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_check_pydevconsole' ___ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_get_referrers.py ____ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_get_referrers.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_get_referrers' ______ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_jyserver.py ______ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_jyserver.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_jyserver' ___ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_jysimpleTipper.py ___ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_jysimpleTipper.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_jysimpleTipper' _ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_pydev_ipython_011.py __ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_pydev_ipython_011.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_pydev_ipython_011' ____ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_pydevconsole.py ____ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_pydevconsole.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_pydevconsole' ______ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_pyserver.py ______ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_pyserver.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_pyserver' ____ ERROR collecting src/ptvsd/_vendored/pydevd/tests/test_simpleTipper.py ____ ImportError while importing test module '$HOME/ptvsd/src/ptvsd/_vendored/pydevd/tests/test_simpleTipper.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: ModuleNotFoundError: No module named 'tests.test_simpleTipper' =============================== warnings summary =============================== $HOME/.local/lib/python3.7/site-packages/_pytest/config/__init__.py:538 $HOME/.local/lib/python3.7/site-packages/_pytest/config/__init__.py:538: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: tests_python.debugger_fixtures self.import_plugin(import_spec) $HOME/.local/lib/python3.7/site-packages/jinja2/utils.py:485 $HOME/.local/lib/python3.7/site-packages/jinja2/utils.py:485: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import MutableMapping $HOME/.local/lib/python3.7/site-packages/jinja2/runtime.py:318 $HOME/.local/lib/python3.7/site-packages/jinja2/runtime.py:318: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Mapping -- Docs: https://docs.pytest.org/en/latest/warnings.html !!!!!!!!!!!!!!!!!!! Interrupted: 8 errors during collection !!!!!!!!!!!!!!!!!!!! ===================== 3 warnings, 8 error in 1.10 seconds ====================== Traceback (most recent call last): File "$HOME/.vscode-server/extensions/ms-python.python-2019.7.27777-dev/pythonFiles/testing_tools/run_adapter.py", line 18, in main(tool, cmd, subargs, toolargs) File "$HOME/.vscode-server/extensions/ms-python.python-2019.7.27777-dev/pythonFiles/testing_tools/adapter/__main__.py", line 90, in main parents, result = run(toolargs, **subargs) File "$HOME/.vscode-server/extensions/ms-python.python-2019.7.27777-dev/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 36, in discover raise Exception('pytest discovery failed (exit code {})'.format(ec)) Exception: pytest discovery failed (exit code 2) ```

Running pytest directly works fine with no args:

PYTHONPATH=src python3 -m pytest --collect-only

However, it has exactly the same failure as the adapter if I pass the same args:

PYTHONPATH=src /usr/bin/python3.7 -m pytest --collect-only -s --cache-clear .

ericsnowcurrently commented 5 years ago

Also, if I drop the final "." then both pytest and the adapter work fine:

PYTHONPATH=src /usr/bin/python3.7 -m pytest --collect-only -s --cache-clear

ericsnowcurrently commented 5 years ago

@karthiknadig, things should work fine with the following settings.json:

{
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": []
}

The following will work too:

{
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--ignore=src/ptvsd/_vendored/",
        "."
    ]
}

I've verified that both work and that both exclude pydevd tests. I'm not sure why that plain "." (the default) causes a problem.

karthiknadig commented 5 years ago

Confirming that this works now.

ericsnowcurrently commented 5 years ago

thanks @karthiknadig

microsoft / vscode-python

Test discovery failed for pytest 4.3.0 (not including cwd in path) #6417

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

Prescribed Solution