pytest fails module not found, even tho tests are discovered & works in debug,

[mikofski]

Thank you for your help! I have search many issues, and tried many solutions, but I can't figure it out. I understand this is OSS, and no one may have time or interest to help me. That's okay. I am very grateful for vscode-python, and appreciate your time.

We have written the needed data into your cl

• VS Code Version: 1.53.2
• OS Version: Windows 10.0.17763
• Python: 3.6.9 from python.org, running in venv
• pytest 6.2.2

Version: 1.53.2 (user setup) Commit: 622cb03f7e070a9670c94bae1a45d78d7181fbd4 Date: 2021-02-11T11:48:04.245Z Electron: 11.2.1 Chrome: 87.0.4280.141 Node.js: 12.18.3 V8: 8.7.220.31-electron.0 OS: Windows_NT x64 10.0.17763

Steps to Reproduce:

1. use python 3.6 to create a virtualenv: $py -3.6 -m venv venv
2. install requirements for pvlib and install pvlib in editable mode: pip install --no-deps -e .
3. open pvlib folder in vscode, it chooses venv automatically, yay!
4. see test are all discovered
5. try to run tests, select pytest as test runner, use default setup.cfg as only setting
6. pick any test and run it, but it fails to import pvlib in conftest.py

``` ImportError while loading conftest 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\conftest.py'. pvlib\tests\conftest.py:10: in import pvlib E ModuleNotFoundError: No module named 'pvlib' Error: TypeError: Cannot read property 'testsuites' of null ```

7. run the same test again using debug, and it works!
8. try it in the console or a windows CMD, and it works! (venv) $ pytest pvlib/tests/test_infinite_sheds.py::test_gcr_prime

Does this issue occur when all extensions are disabled?: Yes/No

If I disable extensions, then the Python extension isn't loaded, and so no tests are discovered, and so I can't run them.

I have the following extensions installed and enabled: Python, Remote-WSL, Jupyter, Git History, C#, and C/C++

If I disable everything but Python and Jupyter (I cannot disable only Jupyter), the tests are still discovered, but still do not run.

Related Issues

• https://github.com/microsoft/vscode-python/issues/14570

[mikofski]

FYI: I've tried the latest insider's build of vscode and the python extension and set "python.useIsolation": false in my .vscode/settings.json and the same error still occurs.

However, if I re-install pvlib using python setup.py install now it works. So I cannot tell if this is a duplicate of #14570 or not. My guess is that it is a duplicate, but then I don't understand why using the insider's version breaks it.

I'd also like to know how others are using pytest and virtualenvs in vscode?

[ericsnowcurrently]

@mikofski, thanks for letting us know about this. Please provide the content of the "Python" output panel and the "Python Test Log" output panel. Also, please provide the content of your settings.json file. Of particular relevance is the "python.testing.pytestArgs" setting.

Another setting to try is "python.testing.cwd". Set it to the root directory from which you want to run your tests. Normally this is the workspace root.

Finally, what happens when you run pytest manually in a terminal? What about if you run the same command that shows up in the "Python" output panel? The command output will be helpful.

[mikofski]

Thank you very much!!!

Here is the Python test log output:

ImportError while loading conftest 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\conftest.py'.
pvlib\tests\conftest.py:10: in <module>
    import pvlib
E   ModuleNotFoundError: No module named 'pvlib'
Error: TypeError: Cannot read property 'testsuites' of null

I think my settings.json file is like this, it was auto generated:

{
    "python.testing.pytestArgs": [],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.pythonPath": "venv\\Scripts\\python.exe"
}

Pytest works fine when I run it manually in a terminal or from the vscode Python output. And discover tests works and tests run in debug. It's only the green play button on the GUI that raises the exception.

I'm away from my laptop, but I will try setting python.testing.cwd asap. Thanks!

[mikofski]

The python extension updated to 2021.2.1 (19 February 2021) v2021.2.582707922 but unfortunately, the tests are still not working.

the Python test log output:

python c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\mikm\Projects\pvlib-python -s --cache-clear
ImportError while loading conftest 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\conftest.py'.
pvlib\tests\conftest.py:10: in <module>
    import pvlib
E   ModuleNotFoundError: No module named 'pvlib'
Error: TypeError: Cannot read property 'testsuites' of null

the python output:

``` User belongs to experiment group 'pythonaa' User belongs to experiment group 'pythonSendEntireLineToREPL' User belongs to experiment group 'pythonNotDisplayLinterPrompt' User belongs to experiment group 'ShowPlayIcon - start' User belongs to experiment group 'ShowExtensionSurveyPrompt - control' User belongs to experiment group 'DebugAdapterFactory - experiment' User belongs to experiment group 'PtvsdWheels37 - experiment' User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control' User belongs to experiment group 'LocalZMQKernel - experiment' User belongs to experiment group 'CollectLSRequestTiming - control' User belongs to experiment group 'CollectNodeLSRequestTiming - experiment' User belongs to experiment group 'EnableIPyWidgets - experiment' User belongs to experiment group 'RunByLine - experiment' User belongs to experiment group 'CustomEditorSupport - control' > conda --version > pyenv root > python3.7 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > python3.6 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > python3 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > python2 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > python c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > py -3.7 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > py -3.6 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > py -3 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > py -2 c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" Error 2021-02-22 20:49:38: Detection of Python Interpreter for Command py and args -2 failed as file does not exist Python interpreter path: .\venv\Scripts\python.exe Starting Jedi Python language engine. > ~\AppData\Local\Continuum\miniconda3\Scripts\conda.exe info --json > ~\AppData\Local\Continuum\miniconda3\Scripts\conda.exe info --json > ~\AppData\Local\Continuum\miniconda3\Scripts\conda.exe info --json > ~\AppData\Local\Continuum\miniconda3\Scripts\conda.exe env list > ~\AppData\Local\Continuum\miniconda3\Scripts\conda.exe info --json > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\mikm\Projects\pvlib-python -s --cache-clear cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.prefix)" cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.prefix)" cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "import sys;print(sys.executable)" cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())" cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())" cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py site --user-site cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py site --user-site cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py pylint --disable=all --enable=F,unreachable,duplicate-key,unnecessary-semicolon,global-variable-not-assigned,unused-variable,unused-wildcard-import,binary-op-exception,bad-format-string,anomalous-backslash-in-string,bad-open-mode,E0001,E0011,E0012,E0100,E0101,E0102,E0103,E0104,E0105,E0107,E0108,E0110,E0111,E0112,E0113,E0114,E0115,E0116,E0117,E0118,E0202,E0203,E0211,E0213,E0236,E0237,E0238,E0239,E0240,E0241,E0301,E0302,E0303,E0401,E0402,E0601,E0602,E0603,E0604,E0611,E0632,E0633,E0701,E0702,E0703,E0704,E0710,E0711,E0712,E1003,E1101,E1102,E1111,E1120,E1121,E1123,E1124,E1125,E1126,E1127,E1128,E1129,E1130,E1131,E1132,E1133,E1134,E1135,E1136,E1137,E1138,E1139,E1200,E1201,E1205,E1206,E1300,E1301,E1302,E1303,E1304,E1305,E1306,E1310,E1700,E1701 --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_atmosphere.py cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py pylint --disable=all --enable=F,unreachable,duplicate-key,unnecessary-semicolon,global-variable-not-assigned,unused-variable,unused-wildcard-import,binary-op-exception,bad-format-string,anomalous-backslash-in-string,bad-open-mode,E0001,E0011,E0012,E0100,E0101,E0102,E0103,E0104,E0105,E0107,E0108,E0110,E0111,E0112,E0113,E0114,E0115,E0116,E0117,E0118,E0202,E0203,E0211,E0213,E0236,E0237,E0238,E0239,E0240,E0241,E0301,E0302,E0303,E0401,E0402,E0601,E0602,E0603,E0604,E0611,E0632,E0633,E0701,E0702,E0703,E0704,E0710,E0711,E0712,E1003,E1101,E1102,E1111,E1120,E1121,E1123,E1124,E1125,E1126,E1127,E1128,E1129,E1130,E1131,E1132,E1133,E1134,E1135,E1136,E1137,E1138,E1139,E1200,E1201,E1205,E1206,E1300,E1301,E1302,E1303,E1304,E1305,E1306,E1310,E1700,E1701 --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text c:\Users\mikm\Projects\pvlib-python\pvlib\tests\test_atmosphere.py cwd: c:\Users\mikm\Projects\pvlib-python > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\completion.py > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\completion.py > ~\AppData\Local\Continuum\miniconda3\Scripts\conda.exe env list ##########Linting Output - pylint########## -------------------------------------------------------------------- Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00) > c:\Users\mikm\Projects\pvlib-python\venv\Scripts\python.exe c:\Users\mikm\.vscode\extensions\ms-python.python-2021.2.582707922\pythonFiles\pyvsc-run-isolated.py pytest --override-ini junit_family=xunit1 --rootdir c:\Users\mikm\Projects\pvlib-python --junit-xml=C:\Users\mikm\AppData\Local\Temp\tmp-14876YDHNsvOYX09o.xml ./pvlib/tests/test_atmosphere.py cwd: c:\Users\mikm\Projects\pvlib-python ```

The settings.json file:

{
    "python.testing.pytestArgs": [],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.pythonPath": "venv\\Scripts\\python.exe"
}

run pytest manually in a terminal?

What about if you run the same command that shows up in the "Python" output panel? The command output will be helpful.

Test discovery works!

Run test in debug mode works!

Unfortunately, setting "python.testing.cwd": "pvlib" in settings.json doesn't alter the outcome. Discover tests still works, debug mode still works, but run tests fails with the same output as before.

So far the only thing that seems to work is to install using python setup.py install but using pip install -e . or python setup.py develop both fail.

I should also note that if create a conda environment (instead of a python virtual environment) I get the same outcome: discovery works, pytest works from terminal, run tests fails with the same error:

ImportError while loading conftest 'c:\Users\mikm\Projects\pvlib-python\pvlib\tests\conftest.py'.
pvlib\tests\conftest.py:10: in <module>
    import pvlib
E   ModuleNotFoundError: No module named 'pvlib'
Error: TypeError: Cannot read property 'testsuites' of null

[wholmgren]

@mikofski I see the same behavior on my mac in my conda environment and having installed pvlib using pip install -e .. It's been awhile since I've used vscode in this way so I didn't notice when it broke (for no particular reason I am in the habit of using vscode's debug test or running from the command line). I'll guess it worked about 3 months ago. Sorry I didn't actually try this when you first reported the issue - I initially understood it to be limited to virtualenv.

[wholmgren]

The relationship to installation issues and a desire to avoid real work led me try installing the package with pyproject.toml and moving the pytest config to that file too. Same issue. Branch here.

[mikofski]

@wholmgren I found a sneaky but unpleasant workaround. I insert this into pvlib/tests/conftest.py right before import pvlib:

import sys
sys.path.append(str(Path(__file__).parent.parent.parent))

If I open a python interpreter and print the Python path the outcome of the "editable" install is that pvlib-python is last in the list:

>>> import pprint
>>> pprint.pprint(sys.path)
['',
 '/usr/lib/python38.zip',
 '/usr/lib/python3.8',
 '/usr/lib/python3.8/lib-dynload',
 '/home/mikm/Projects/pvlib-python/venv/lib/python3.8/site-packages',
 '/home/mikm/Projects/pvlib-python']

but printing out the Python path from inside pvlib/tests/conftest.py:

import pprint
pprint.pprint(sys.path)

shows during test execution it's been removed:

['/home/mikm/Projects/pvlib-python/pvlib/tests',
 '/home/mikm/.vscode-server/extensions/ms-python.python-2021.2.582707922/pythonFiles',
 '/usr/lib/python38.zip',
 '/usr/lib/python3.8',
 '/usr/lib/python3.8/lib-dynload',
 '/home/mikm/Projects/pvlib-python/venv/lib/python3.8/site-packages']

[mhostetter]

FYI: I've tried the latest insider's build of vscode and the python extension and set "python.useIsolation": false in my .vscode/settings.json and the same error still occurs.

I believe you need to set "python.useIsolation": false in your User Settings not your Workspace settings in VS Code Insiders. https://github.com/microsoft/vscode-python/issues/14570#issuecomment-755445657

That worked for me.

[mikofski]

@wholmgren FYI vscode testing is working in other projects, so there is something specific about pvlib that is different.

• conftest.py
• tests/ doesn't have __init__.py
• tests/ isn't at the project level, it's inside the pvlib package

Have you tried the vscode insiders build and setting "python.useIsolation": false in the User Settings? This supposedly fixes the issue, but either I did it wrong or it didn't work for me. I guess setting python.useIsolation to false undoes this line: https://github.com/microsoft/vscode-python/blob/1079ab24353c605629b0308b593543fddc78f7f7/pythonFiles/pyvsc-run-isolated.py#L20

I guess I'm not terribly concerned b/c pytest works in a terminal and that's good enough for me. And visual debug still works in code, so that's nice. I'm kinda hoping this isn't a larger issue, and it just works itself out in time. 😏

[mikofski]

fixed in https://github.com/pvlib/pvlib-python/pull/1204

[mikofski]

related to discussion https://github.com/microsoft/vscode-python/discussions/15494

[FrancescoValerio]

FYI: I've tried the latest insider's build of vscode and the python extension and set "python.useIsolation": false in my .vscode/settings.json and the same error still occurs.

I believe you need to set "python.useIsolation": false in your User Settings not your Workspace settings in VS Code Insiders. #14570 (comment)

That worked for me.

I would like to add that this also solved my bug. This being that pytest or in other words testing does not respect PYTHONPATH when it is in the root directory.

The fix, that is, adding "python.useIsolation": false` to Preferences: Open Settings (JSON) solved the following issue:

0. create a parent directory example_parent
1. create a directory named foo and at the same level create a directory called test
2. within the foo directory create a main.py file containing the following code:

   def return_4():
   return 4

3. within the test directory create a file called test_file.py containing the following code:

   
   from foo.main import return_4

def test_4(): assert return_4() != 4

4. add the parent directory that foo and test reside in to your PYTHONPATH (with Mac this is done through sudo nano /private/etc/zprofile and then add :
```export PYTHONPATH=~/folder/to/example_parent```

5. open vscode
6. within vscode open example_parent
7. press ctrl+shift+p and then type python all tests
8. enable and configure pytest in the root directory
9.  run pytest within vscode through the python run all tests setting with vscode opened in root
10. You will receive the following error

python /Users/francesco/.vscode/extensions/ms-python.python-2021.4.765268190/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /Users/francesco/Work/example_parent -s --cache-clear . Test Discovery failed: Error: ============================= test session starts ============================== platform darwin -- Python 3.8.2, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 rootdir: /Users/francesco/Work/example_parent plugins: xdist-2.2.1, forked-1.3.0 collected 0 items / 1 error

==================================== ERRORS ==================================== _ ERROR collecting tests/test_file.py __ ImportError while importing test module '/Users/francesco/Work/example_parent/tests/test_file.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/importlib/init.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_file.py:2: in from foo.main import return_4 E ModuleNotFoundError: No module named 'foo' =========================== short test summary info ============================ ERROR tests/test_file.py !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!! ===================== no tests collected, 1 error in 0.08s =====================

Traceback (most recent call last): File "/Users/francesco/.vscode/extensions/ms-python.python-2021.4.765268190/pythonFiles/testing_tools/run_adapter.py", line 22, in main(tool, cmd, subargs, toolargs) File "/Users/francesco/.vscode/extensions/ms-python.python-2021.4.765268190/pythonFiles/testing_tools/adapter/main.py", line 100, in main parents, result = run(toolargs, **subargs) File "/Users/francesco/.vscode/extensions/ms-python.python-2021.4.765268190/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover raise Exception("pytest discovery failed (exit code {})".format(ec)) Exception: pytest discovery failed (exit code 2)