Handling of symlinks in pytest discovery

[rjra100]

Type: Bug

Behaviour

I'm finding it difficult to get VSCode to correctly discover pytest tests in a workspace whose path involves a symlink. The output of <python> run_adapter.py discover pytest varies depending exactly how it's called (and the extension reacts differently to these variations), but the Output pane doesn't accurately reflect what inputs were passed to run_adapter.py, making it tricky to work out exactly what's going on.

Ultimately, I'd like to be able to create my workspace in an arbitrary location (i.e. where my IT department provided me with storage - something like /data/store_123456/workspace) but reference it via a symlink from my home directory (so Open Folder /home/<me>), and have Pytest discovery work without my needing to muck about with its inputs.

As a secondary issue, the Output pane should accurately reflect how the test adapter is actually invoked.

Steps to reproduce:

1. Make a directory, put a dummy pytest in it:

   mkdir -p ~/real_location/test
   echo -e "def test_something():\n    pass" > ~/real_location/test/test_example.py

2. Make a symlink to it:

   ln -s ~/real_location~/my_workspace 

3. Open the symlink folder in VSCode (Open Folder, navigate to ~/my_workspace)
4. Activate the Python extension (open the file); open the testing pane and enable pytest testing.

Case 1 : default settings (does not work)

5. Examine the testing pane.
   • Expected behaviour: the test shows up
   • Actual behaviour: the test does not appear (a real_location workspace root node appears, nothing underneath)
     Output pane

2024-08-14 09:06:51.493 [info] Discover tests for workspace name: my_workspace - uri: /home/ralexander72/my_workspace
2024-08-14 09:06:51.525 [warning] could not find a pixi interpreter for the interpreter at /bin/python3
2024-08-14 09:06:51.562 [warning] could not find a pixi interpreter for the interpreter at /bin/python3
2024-08-14 09:06:51.567 [info] > /bin/python3 ~/.vscode-server/extensions/ms-python.python-2024.12.3/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2024-08-14 09:06:51.567 [info] cwd: .

Case 2 : explicit directory (does not work, differently)

6. Set python.testing.pytestArgs to ${workspaceFolder}
7. Examine the testing pane.
   • Expected behaviour: the test shows up and can be run
   • Actual behaviour:
   • my_workspace node shows up in test explorer, containing the test (if run in the same window, the existing real_location node remains)
   • Attempting to run the test (click the run or debug button next to it), nothing happens (Test Results pane shows "Finished running tests!"; the test does not run).

Output pane

``` 2024-08-14 09:09:13.173 [info] Discover tests for workspace name: my_workspace - uri: /home/ralexander72/my_workspace 2024-08-14 09:09:13.173 [warning] could not find a pixi interpreter for the interpreter at /bin/python3 2024-08-14 09:09:13.226 [warning] could not find a pixi interpreter for the interpreter at /bin/python3 2024-08-14 09:09:13.229 [info] > /bin/python3 ~/.vscode-server/extensions/ms-python.python-2024.12.3/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear . 2024-08-14 09:09:13.229 [info] cwd: . ```

Case 3 : explicit directory and rootdir (does work)

8. Set python.testing.pytestArgs to ["--rootdir=.", "${workspaceFolder}"]
9. Examine the testing pane.
   • Expected and actual behaviour: the test shows up and can be run. Phew.

Output pane

``` 2024-08-14 09:13:37.175 [info] Discover tests for workspace name: my_workspace - uri: /home/ralexander72/my_workspace 2024-08-14 09:13:37.176 [warning] could not find a pixi interpreter for the interpreter at /bin/python3 2024-08-14 09:13:37.184 [warning] could not find a pixi interpreter for the interpreter at /bin/python3 2024-08-14 09:13:37.188 [info] > /bin/python3 ~/.vscode-server/extensions/ms-python.python-2024.12.3/python_files/testing_tools/run_adapter.py discover pytest -- -s --cache-clear --rootdir=. . 2024-08-14 09:13:37.188 [info] cwd: ```

I gather the wittering about pixi is harmless noise (though it'd be nice if that could be shut up). There is no further output displayed: no error in any case; things just don't work quite right.

Note that despite passing ${workspaceFolder} (i.e. a full path ~/my_workspace), the output pane consistently substitutes the workspace directory with ., making it impossible to distinguish --rootdir=. from --rootdir=${workspaceFolder}, for example. Note that for Pytest discovery, this difference is significant.

Analysis

What I think's happening depends on exactly what arguments are really passed to run_adapter.py discover pytest. Adding pretty formatting to the output for readability:

python <...>/run_adapter.py discover pytest --pretty -- --rootdir ~/my_workspace -s --cache-clear

``` [ { "parents": [ { "id": "./test", "kind": "folder", "name": "test", "parentid": ".", "relpath": "./test" }, { "id": "./test/test_example.py", "kind": "file", "name": "test_example.py", "parentid": "./test", "relpath": "./test/test_example.py" } ], "root": "/home/ralexander72/real_location", "rootid": ".", "tests": [ { "id": "./test/test_example.py::test_something", "markers": [], "name": "test_something", "parentid": "./test/test_example.py", "source": "./../real_location/test/test_example.py:1" } ] } ] ```

Note: "root": "/home/ralexander72/real_location"; "source": "./../real_location/test/test_example.py:1"

I suspect this is what's actually being invoked by default (--rootdir . in the output pane notwithstanding), and that the extension sees the file locations as somewhere outside of the workspace (/home/ralexander72/my_workspace) and ignores the tests.

python <...>/run_adapter.py discover pytest --pretty -- --rootdir ~/my_workspace -s --cache-clear ~/my_workspace

``` [ { "parents": [ { "id": "./test", "kind": "folder", "name": "test", "parentid": ".", "relpath": "./test" }, { "id": "./test/test_example.py", "kind": "file", "name": "test_example.py", "parentid": "./test", "relpath": "./test/test_example.py" } ], "root": "/home/ralexander72/my_workspace", "rootid": ".", "tests": [ { "id": "./test/test_example.py::test_something", "markers": [], "name": "test_something", "parentid": "./test/test_example.py", "source": "./../real_location/test/test_example.py:1" } ] } ] ```

Note: "root": "/home/ralexander72/my_workspace"; "source": "./../real_location/test/test_example.py:1"

I suspect this is what's actually being invoked with "python.testing.pytestArgs" : [ "${workspaceFolder}" ]. Now the root path is sane, and the tests are therefore considered part of the workspace and show up. However, the source paths are "relative" paths from the symlinked path to the "real" one - opening the test via the "Go To Test" button shows the file path as /home/ralexander72/real_location/test/test_example.py, and the test won't run.

python <...>/run_adapter.py discover pytest --pretty -- --rootdir . -s --cache-clear ~/my_workspace

``` [ { "parents": [ { "id": "./test", "kind": "folder", "name": "test", "parentid": ".", "relpath": "./test" }, { "id": "./test/test_example.py", "kind": "file", "name": "test_example.py", "parentid": "./test", "relpath": "./test/test_example.py" } ], "root": "/home/ralexander72/my_workspace", "rootid": ".", "tests": [ { "id": "./test/test_example.py::test_something", "markers": [], "name": "test_something", "parentid": "./test/test_example.py", "source": "./test/test_example.py:1" } ] } ] ```

Note: "root": "/home/ralexander72/my_workspace"; "source": "./test/test_example.py:1"

These arguments are now as passed explicitly on the command line ("python.testing.pytestArgs" : [ "--rootdir=.", "${workspaceFolder}" ] - though the output pane still shows --rootdir=. .) Now the root and source paths both look sensible, and configured this way, the test both shows up and works.

Version info

Extension version: 2024.12.2, 2024.12.3 (seems to have updated overnight!) VS Code version: Code 1.92.1 (eaa41d57266683296de7d118f574d0c2652e1fc4, 2024-08-07T20:16:39.455Z) OS version: Windows_NT x64 10.0.19045 Modes: Remote OS version: Linux x64 5.15.153.1-microsoft-standard-WSL2 - Python version: version-independent, but tried 3.10.12+Pytest 8.2, 3.12.4+Pytest 8.1.1 - Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): None - Value of the `python.languageServer` setting: Default

User Settings

``` languageServer: "Pylance" testing • pytestArgs: "" or "${workspaceFolder}; see analysis above • pytestEnabled: true ```

Installed Extensions

|Extension Name|Extension Id|Version| |---|---|---| |Auto Snippet|Gruntfuggly.auto-snippet|0.0.11| |Black Formatter|ms-python.black-formatter|2024.2.0| |C/C++|ms-vscode.cpptools|1.21.6| |clangd|llvm-vs-code-extensions.vscode-clangd|0.1.29| |CMake|twxs.cmake|0.0.17| |CMake Test Explorer|fredericbonnet.cmake-test-adapter|0.17.4| |CMake Tools|ms-vscode.cmake-tools|1.18.44| |CodeLLDB|vadimcn.vscode-lldb|1.10.0| |Command Variable|rioj7.command-variable|1.63.0| |GitLens — Git supercharged|eamodio.gitlens|15.2.3| |JavaScript Debugger|ms-vscode.js-debug|1.92.0| |Live Preview|ms-vscode.live-server|0.4.14| |markdownlint|DavidAnson.vscode-markdownlint|0.55.0| |Pylance|ms-python.vscode-pylance|2024.8.1| |Python|ms-python.python|2024.12.2| |Python Debugger|ms-python.debugpy|2024.10.0| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.9| |Test Adapter Converter|ms-vscode.test-adapter-converter|0.1.9| |Test Explorer UI|hbenl.vscode-test-explorer|2.21.1| |Time Travel Debug for C/C++|undo.udb|2.0.4|

[eleanorjboyd]

Hello! I think this issue is fixed by opting into this experiment. Give it a try and let me know! Add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"] then try again. You can confirm you have the rewrite enabled by setting log levels to trace (via command palette) then check for Experiment 'pythonTestAdapter' is active in your python logs.

Let me know if the rewrite fixes your issue. Thanks!