Extension v2024.4.0 and v2024.4.1 create indefinite "Discover Python Interpreters"

[dokutoshi]

Type: Bug

Behaviour

Expected vs. Actual

Install v2024.4.0 or v2024.4.1 extension and the discovering Python interpreters run indefinitely. This also restarts the kernel periodically - a MAJOR PROBLEM when excecuting ML notebooks - lost variables imply retraining or reloading a pandas DF etc. XXX

Steps to reproduce:

To stop the kernel from periodic restarts use v2024.2.1

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.11.9
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Conda
• Value of the python.languageServer setting: Pylance

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` XXX ```

User Settings

``` languageServer: "Pylance" ```

Extension version: 2024.2.1 VS Code version: Code 1.88.0 (5c3e652f63e798a5ac2f31ffd0d863669328dc4c, 2024-04-03T13:28:09.495Z) OS version: Darwin arm64 23.5.0 Modes:

System Info

|Item|Value| |---|---| |CPUs|Apple M1 Pro (10 x 24)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|5, 5, 5| |Memory (System)|16.00GB (0.05GB free)| |Process Argv|--crash-reporter-id aaabbd8f-9170-4ae6-b93a-f11487600abe| |Screen Reader|no| |VM|0%|

A/B Experiments

``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 tftest:31042121 vswsl492cf:30256860 vscod805cf:30301675 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialogc:30910334 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pythonmypyd1:30879173 pythoncet0:30885854 2e7ec940:31000449 pythontbext0:30879054 accentitlementst:30995554 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 chatpanelc:31048052 dsvsc021:30996838 9c06g630:31013171 fchga941:31046352 pythonprc:31047982 dwnewjupytercf:31046870 26j00206:31048877 ```

[anthonykim1]

Hi @dokutoshi. Thank you for filing the issue. Can you set your log level to trace in command palette and share your Python output info accessible in output for Python?

[dcx]

I'm having this issue too, and there's nothing available in the Output dialog.

[anthonykim1]

Interesting..Does Clear Cache and reload Python extension fix the problem? (Accessible via ctrl+shift+p) or cmd+shift+p on mac.

[anthonykim1]

Also wondering if problem persists in version below v2024.4.0

[dokutoshi]

I will look into this later today.

Prescriptive measure include setting log level == 'trace' and cmd+shift+p

On May 22, 2024, at 14:49, Anthony Kim @.***> wrote:

Also wondering if problem persists in version below v2024.4.0

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode-python/issues/23452#issuecomment-2125526612, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACNWBJOTDMYMQLMVSSOMG6TZDTSCVAVCNFSM6AAAAABH63ATVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRVGUZDMNRRGI. You are receiving this because you were mentioned.Message ID: @.***>

This email and any attachments may contain private, confidential and privileged material for the sole use of the intended recipient. Any review, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and delete this email and any attachments afterwards.

[dokutoshi]

Clearing the cache and reloading the window does solve the infinite running issue.

However, the reason i opened this ticket relates to this bug:

48 hours into testing, Discovering process starts up and terminates a running job. ML training takes weeks. What happened? Using VSCode worked for over 2 years in DL training. Now I can barely use it for a day without Discovering Python Interpreters appearing and halting training. @anthonykim1

[anthonykim1]

@dokutoshi Can you bring the requested logs and also check if same problem exists in version below 2024.4.0 that you have mentioned?

[dokutoshi]

Dealing with a related issue in Vscode Insiders that crashes Pylance and triggers Discovering Python Interpreters. note the error using the -I flag in the Vscode code base. Pylance Crash

For reference, here is the Python log output.

2024-06-04 18:31:35.095 [info] shell: bash 2024-06-04 18:31:35.732 [info] > /usr/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py 2024-06-04 18:31:35.802 [info] > ./miniforge-pypy3/envs/tf_2.15/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py

2024-06-04 18:31:35.816 [warning] [Error: Command failed: /usr/bin/python -I /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py Unknown option: -I usage: /usr/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ... Try `python -h' for more information.

at ChildProcess.exithandler (node:child_process:423:12)
at ChildProcess.emit (node:events:530:35)
at maybeClose (node:internal/child_process:1105:16)
at Socket.<anonymous> (node:internal/child_process:457:11)
at Socket.emit (node:events:518:28)
at Pipe.<anonymous> (node:net:337:12)] {

code: 2, killed: false, signal: null, cmd: '/usr/bin/python -I /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py /home/ec2-user/.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py' } 2024-06-04 18:31:35.816 [error] Support for Python 2.7 has been dropped by the Python extension so certain features may not work, upgrade to using Python 3. 2024-06-04 18:31:35.843 [info] > /usr/bin/python ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py 2024-06-04 18:31:36.115 [info] > ./miniforge-pypy3/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py 2024-06-04 18:31:36.279 [info] > /usr/local/bin/python -I ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/get_output_via_markers.py ./.vscode-server-insiders/extensions/ms-python.python-2024.6.0/python_files/interpreterInfo.py 2024-06-04 18:31:36.554 [info] Starting Pylance language server.

[dokutoshi]

Sorry venting here but when the Discovery process starts, it is impossible to save the file.

Even Save As fails silently!

[dokutoshi]

For clarification, the Discovering Python Interpreters occurs from Python 2024.2.1 version up to and including the most recent version of VsCode Insiders. @anthonykim1 - The answer is Yes!

Both local (Mac) and remote SSH (from Mac to Linux) exhibit the issues.

Version: 1.84.2 Commit: 1a5daa3a0231a0fbba4f14db7ec463cf99d7768e Date: 2023-11-09T10:52:57.054Z Electron: 25.9.2 ElectronBuildId: 24603566 Chromium: 114.0.5735.289 Node.js: 18.15.0 V8: 11.4.183.29-electron.0 OS: Darwin arm64 23.3.0 Python: Extension version: 2024.2.1

Ahh, ran extension profiler.

https://github.com/microsoft/vscode-python/issues/23554

[anthonykim1]

@dokutoshi Thanks for the requested logs. Are you using Python2.7? I see

 [error] Support for Python 2.7 has been dropped by the Python extension so certain features may not work, upgrade to using Python 3.

from your log.

Support for 2.7 has been dropped by Python extension. Reference: https://github.com/microsoft/vscode-python/wiki/Limited-support-for-Python-2.7

[dokutoshi]

Python

No, I use python 3.10 - 3.12.

Conda environments established on the ec2. Why is vscode searching the ec2 for possible python directories outside of the designated environment?

Best,

Leo

[dokutoshi]

Btw, is there a method to suppress the discovery process e..g "Discovering Python Interpreters". The process appears event or timer based. is there a method to make this process interactive i.e. user controlled?

[anthonykim1]

Conda environments established on the ec2. Why is vscode searching the ec2 for possible python directories outside of the designated environment?

It is not necessarily VS Code, it would be Python extension or other extension IN VS Code that is triggering interpreter discovery.

Btw, is there a method to suppress the discovery process e..g "Discovering Python Interpreters". The process appears event

I don't think so. The problem you are having can happen when some other extension triggers discovery, or there is some changes in the Python environment that you have set up. For now you can pin the extension to older version where everything used to work for you until we investigate this further.

I will keep this issue open just so I can get a census/reports of to the extent other users are experiencing the issue.

[dokutoshi]

i launched a new EC2 - Ubuntu 22.04 built on Python3 to avoid the interrupt encountered with Python 2.7.

Something is triggering the discovery and... the designated kernel is no longer stable. I use only Microsoft extensions.

This is problematic.

[dokutoshi]

Maybe the Python Debugger extension @anthonykim1

[dokutoshi]

Figured out the bug:

Here is the latest @anthonykim1 ... it appears VScode's Python extension joins the local python environments and remote python environment directories, perhaps into a list of paths.
If the directories are the same on the local and remote, no issue.
If there are paths on the local, not present on the remote, the python extension crashes as the Discover Python Interpreters enters an infinite loop.

Remote device: Python Log 2024-06-09 17:41:28.668 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Remote device: `ubuntu@ip-172-31-92-108:/mnt_data/home/ec2-user$ which -a python python2 python3

/usr/bin/python3 /bin/python3`

Local device: `ec2-user@ip-172-31-0-196 homebrew % which -a python python2 python3 python not found

python2 not found

/opt/homebrew/bin/python3 /usr/bin/python3`

Setup:

1. An AWS Mac acts as a bastion device to remote ssh to a Linux device. [Not sure that matters]

2. The Python extension uses scandir on the remote Linux device and on the localhost device. It then searches for all the absolute paths in the list on the remote device irrespective if the path is only on the local machine. @anthonykim1

   Temporary Solution: `# Create the necessary directory if it doesn't exist sudo mkdir -p /opt/homebrew/bin

Create the symbolic link

sudo ln -s /usr/bin/python3 /opt/homebrew/bin/python3

Verify the symlink

ls -l /opt/homebrew/bin/python3`

Result of Python discovery: Pre-fix: 2024-06-09 17:50:36.354 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Post-fix: 2024-06-09 17:55:57.165 [info] Experiment 'pythonCreateEnvOnPipInstall' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonRecommendTensorboardExt' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonSurveyNotification' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonTerminalEnvVarActivation' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonTestAdapter' is active 2024-06-09 17:55:57.165 [info] Default formatter is set to ms-python.black-formatter for workspace /mnt_data/home/ec2-user 2024-06-09 17:55:57.165 [info] Starting Environment refresh 2024-06-09 17:55:57.165 [info] Searching for interpreters in posix paths locator 2024-06-09 17:55:57.165 [info] Searching for pyenv environments 2024-06-09 17:55:57.165 [info] Searching for conda environments 2024-06-09 17:55:57.165 [info] Searching for global virtual environments 2024-06-09 17:55:57.165 [info] Searching for custom virtual environments 2024-06-09 17:55:57.165 [info] pyenv is not installed 2024-06-09 17:55:57.165 [info] Finished searching for pyenv environments: 339 milliseconds 2024-06-09 17:55:57.165 [info] Finished searching for custom virtual envs: 339 milliseconds 2024-06-09 17:55:57.165 [info] > conda info --json 2024-06-09 17:55:57.165 [info] > hatch env show --json 2024-06-09 17:55:57.165 [info] cwd: . 2024-06-09 17:55:57.165 [info] Finished searching for global virtual envs: 361 milliseconds 2024-06-09 17:55:57.287 [info] Found: /bin/python3 --> /bin/python3.10 2024-06-09 17:55:57.368 [info] Found: /bin/python3.10 --> /bin/python3.10 2024-06-09 17:55:57.444 [info] Found: /usr/bin/python3 --> /usr/bin/python3.10 2024-06-09 17:55:57.449 [info] Found: /usr/bin/python3.10 --> /usr/bin/python3.10 2024-06-09 17:55:57.457 [info] > /opt/homebrew/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:57.487 [info] Python interpreter path: /opt/homebrew/bin/python3 2024-06-09 17:55:57.503 [info] Found: /home/ubuntu/miniforge3/bin/python --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.509 [info] Found: /home/ubuntu/miniforge3/bin/python3 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.510 [info] Found: /home/ubuntu/miniforge3/bin/python3.1 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.515 [info] Found: /home/ubuntu/miniforge3/bin/python3.10 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.549 [info] > /bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:57.598 [info] Finished searching for interpreters in posix paths locator: 793 milliseconds 2024-06-09 17:55:57.598 [info] > /usr/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:58.193 [info] Finished searching for conda environments: 1389 milliseconds

[dokutoshi]

There is still an issue... it is not possible to save the file while the Discovery process executes.

[dokutoshi]

Is there an approach to limit the Python environment to a specific path e.g. in a workspace environment file?

{ "python.venvPath": "/path/to/your/allowed/venvs", "python.autoComplete.extraPaths": [ "/path/to/your/allowed/venvs" ] }

[dokutoshi]

Really appreciate some help here @anthonykim1 Several articles of users experiencing similar issues:

1. https://stackoverflow.com/questions/78251441/visual-studio-code-keeps-discovering-python-interpreters-forever-and-vscode-serv
2. https://stackoverflow.com/questions/71153859/discovering-python-interpreters-taking-infinite-time-in-vs-code
3. https://github.com/microsoft/vscode-jupyter/issues/12974 @DonJayamanne
4. https://github.com/microsoft/vscode-python/issues/21310 - Recall there is a symbolic link as VScode searches for the local path on the remote machine. ((
5. https://github.com/microsoft/vscode-python/issues/20250

Issues were not solved. I suggest a look at the possible solution.

Figured out the bug:

Here is the latest @anthonykim1 ... it appears VScode's Python extension joins the local python environments and remote python environment directories, perhaps into a list of paths. If the directories are the same on the local and remote, no issue. If there are paths on the local, not present on the remote, the python extension crashes as the Discover Python Interpreters enters an infinite loop.

Remote device: Python Log 2024-06-09 17:41:28.668 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Remote device: `ubuntu@ip-172-31-92-108:/mnt_data/home/ec2-user$ which -a python python2 python3

/usr/bin/python3 /bin/python3`

Local device: `ec2-user@ip-172-31-0-196 homebrew % which -a python python2 python3 python not found

python2 not found

/opt/homebrew/bin/python3 /usr/bin/python3`

Setup:

1. An AWS Mac acts as a bastion device to remote ssh to a Linux device. [Not sure that matters]
2. The Python extension uses scandir on the remote Linux device and on the localhost device. It then searches for all the absolute paths in the list on the remote device irrespective if the path is only on the local machine. @anthonykim1

Temporary Solution: `# Create the necessary directory if it doesn't exist sudo mkdir -p /opt/homebrew/bin

Create the symbolic link

sudo ln -s /usr/bin/python3 /opt/homebrew/bin/python3

Verify the symlink

ls -l /opt/homebrew/bin/python3`

Result of Python discovery: Pre-fix: 2024-06-09 17:50:36.354 [warning] Identifier for virt-virtualenv failed to identify /opt/homebrew/bin/python3 [Error: ENOENT: no such file or directory, scandir '/opt/homebrew/bin'] { errno: -2, code: 'ENOENT', syscall: 'scandir', path: '/opt/homebrew/bin' }

Post-fix: 2024-06-09 17:55:57.165 [info] Experiment 'pythonCreateEnvOnPipInstall' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonRecommendTensorboardExt' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonSurveyNotification' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonTerminalEnvVarActivation' is active 2024-06-09 17:55:57.165 [info] Experiment 'pythonTestAdapter' is active 2024-06-09 17:55:57.165 [info] Default formatter is set to ms-python.black-formatter for workspace /mnt_data/home/ec2-user 2024-06-09 17:55:57.165 [info] Starting Environment refresh 2024-06-09 17:55:57.165 [info] Searching for interpreters in posix paths locator 2024-06-09 17:55:57.165 [info] Searching for pyenv environments 2024-06-09 17:55:57.165 [info] Searching for conda environments 2024-06-09 17:55:57.165 [info] Searching for global virtual environments 2024-06-09 17:55:57.165 [info] Searching for custom virtual environments 2024-06-09 17:55:57.165 [info] pyenv is not installed 2024-06-09 17:55:57.165 [info] Finished searching for pyenv environments: 339 milliseconds 2024-06-09 17:55:57.165 [info] Finished searching for custom virtual envs: 339 milliseconds 2024-06-09 17:55:57.165 [info] > conda info --json 2024-06-09 17:55:57.165 [info] > hatch env show --json 2024-06-09 17:55:57.165 [info] cwd: . 2024-06-09 17:55:57.165 [info] Finished searching for global virtual envs: 361 milliseconds 2024-06-09 17:55:57.287 [info] Found: /bin/python3 --> /bin/python3.10 2024-06-09 17:55:57.368 [info] Found: /bin/python3.10 --> /bin/python3.10 2024-06-09 17:55:57.444 [info] Found: /usr/bin/python3 --> /usr/bin/python3.10 2024-06-09 17:55:57.449 [info] Found: /usr/bin/python3.10 --> /usr/bin/python3.10 2024-06-09 17:55:57.457 [info] > /opt/homebrew/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:57.487 [info] Python interpreter path: /opt/homebrew/bin/python3 2024-06-09 17:55:57.503 [info] Found: /home/ubuntu/miniforge3/bin/python --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.509 [info] Found: /home/ubuntu/miniforge3/bin/python3 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.510 [info] Found: /home/ubuntu/miniforge3/bin/python3.1 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.515 [info] Found: /home/ubuntu/miniforge3/bin/python3.10 --> /home/ubuntu/miniforge3/bin/python3.10 2024-06-09 17:55:57.549 [info] > /bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:57.598 [info] Finished searching for interpreters in posix paths locator: 793 milliseconds 2024-06-09 17:55:57.598 [info] > /usr/bin/python3 -I ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.8.0/python_files/interpreterInfo.py 2024-06-09 17:55:58.193 [info] Finished searching for conda environments: 1389 milliseconds

[dokutoshi]

Turned on Trace log

1. Why is VS Code looking for node_modules under the home directory e.g. ~/node_modules?
2. What is setActiveFolders? Related open ticket: https://github.com/microsoft/vscode-remote-release/issues/7178

Hypothesis: Some disconnect occurs during Remote SSH or Tunnels. I guess the reconnection triggers the Python Discovery process @anthonykim1

AWS support recommended this ticket is related: https://github.com/microsoft/vscode-remote-release/issues/9832

These errors occur for both Remote SSH or Remote Tunnels.

Errors

**2024-06-12 05:31:40.234 [error] command 'remote-internal.setActiveFolders' not found: Error: command 'remote-internal.setActiveFolders' not found at v.n (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1637:23867)** at v.executeCommand (vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1637:23796) 2024-06-12 05:31:41.198 [warning] [remote-ssh]: Couldn't find message for key setting.usseExecServer.description. 2024-06-12 05:31:41.248 [trace] lifecycle: phase changed (value: 4) 2024-06-12 05:31:41.274 [trace] ForwardedPorts: Doing first restore. 2024-06-12 05:31:41.288 [trace] ForwardedPorts: (TunnelModel) restoring ports 6006 **2024-06-12 05:31:41.310 [trace] Error: ENOENT: no such file or directory, stat '/mnt_data/home/ec2-user/node_modules':** EntryNotFound (FileSystemError): Error: ENOENT: no such file or directory, stat '/mnt_data/home/ec2-user/node_modules' at k.create (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:114:97841) at I (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:114:97955) at t.hb (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:119:35958) at t.stat (/home/ubuntu/.vscode-server/bin/89de5a8d4d6205e5b11647eb6a74844ca23d2573/out/vs/server/node/server.main.js:119:28306)

[dokutoshi]

Please take a look at this @karthiknadig . I believe it relates to concerns in this ticket. https://github.com/microsoft/vscode-python/pull/21639 The process never terminates on a remote server. Steps to recreate

1. Connect to a remote server
2. Command Palette - select Python interpreter
3. Observe Discover Python Interpreters start
4. Select interpreter or close it.
5. Observe Discover Python Interpreters does not terminate

[dokutoshi]

May hint at the a test scenario @karthiknadig

1. Visual Studio Code keeps discovering python interpreters forever and vscode-server on remote is busy 100%

Solution proposed on Mar 31, 2024 It is common for computer vision scientists to have directories with millions of images. I will consider moving the directory, this can take 1-2 days for 4 million images. This is not preferred for many reasons such as updating code base etc.

Eventually the remote SSH is dropped because the Discovery Crashes the IDE. Then the process starts over again. Extremely frustrating.

2. "Discovering Python Interpreters" taking Infinite time in VS Code Unsolved

3. Stuck at discovering Python Interpreters in ssh connection Ticket closed but unsolved

[dokutoshi]

Solved! Followed the answer to Number 1. I moved the notebook and other code source files to another project. The image directory with millions of files is outside of the project folder.

Why is the discovery process scanning all the subdirectories???? @karthiknadig @DonJayamanne @anthonykim1

[karthiknadig]

It looks for virtual environments created for monorepos this way. As I said we are re-doing this to be more efficient. But, basically some projects have several virtual environments for each sub folder, that we need to support. We are changing this to handle cases like yours better.

[dokutoshi]

@karthiknadig it is far simpler.
@DonJayamanne @anthonykim1

I was able to recreate the issue locally.

1. Set up a project with a data folder, src folder.
2. Data folder create 1,000,000 images, one can probably generate doc1.txt - doc999999.txt.
3. In the src folder, add python file and jupyter notebook, keep it simple.
4. Select interpreter at project level. Note the time it takes for the kernel to stop spinning. It can be at least 1 - 2 minutes. Sometimes i get a cup of tea.
5. Close folder and reopen at src folder. Vola, select interpreter. It is instant!

I was able to recreate this on a few machines - local and remote are both faster when using only the src folder and not including the data folder (1,000,000+ images, files, etc)

Best,

Leo

[dokutoshi]

One final question - @karthiknadig @anthonykim1. Why not allow the "folders to exclude" feature to limit the search path for the python interpreters or better yet, allow the user to customize? It is not advised for the sentinel to search across a device; there are corporate concerns with such activity that will prevent the use of the tool.

[karthiknadig]

We do have plans on adding customizable excludes in the new work that is being done for environment discovery. But the old discovery will likely not get update to add the new features to it.

[dokutoshi]

Is the current solution to limit the number of files inside a workspace e.g. remove large data i.e. image or corpus directories?

[karthiknadig]

Unfortunately yes.

[dokutoshi]

@karthiknadig - Does Visual Studio have the same limitations i.e. python interpreter scans all files in workspace?

ahh, not a long term solution anyhow... https://learn.microsoft.com/en-us/visualstudio/mac/what-happened-to-vs-for-mac?view=vsmac-2022

[karthiknadig]

@dokutoshi For long term we are going to switch to a different mechanism for discovery and one of the things we are seeing there is providing a way to exclude certain directories, in addition to improving the search performance.

[dokutoshi]

@karthiknadig that would be most excellent. From a corporate perspective, the current limitation created the following situation:

1. Migrate all AWS Linux 2 servers to Ubuntu as Linux 2 contains a python 2.7 env, albeit unused.
2. Load only the source directory. Exclude Directories such as TF folders, parquet and csv data source, and other meta libraries from the workspace. In addition, data folders were included.

These are significant changes at a corporate level . Ubuntu brings its own inherent limitations non-XFS etc that required further steps to support ML workflows.