Error when trying to detect environment

[karthiknadig]

Traceback (most recent call last):
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\inference\compiled\subprocess\__init__.py", line 249, in _send
    is_exception, traceback, result = pickle_load(self._get_process().stdout)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\_compatibility.py", line 12, in pickle_load
    return pickle.load(file)
EOFError: Ran out of input

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\api\environment.py", line 77, in _get_subprocess
    info = self._subprocess._send(None, _get_info)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\inference\compiled\subprocess\__init__.py", line 257, in _send
    raise InternalError(
jedi.api.exceptions.InternalError: The subprocess C:\Program Files (x86)\wapt\Scripts\python.exe has crashed (EOFError('Ran out of input'), stderr=).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\pygls\protocol.py", line 340, in _handle_request
    self._execute_request(msg_id, handler, params)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\pygls\protocol.py", line 264, in _execute_request
    self._send_response(msg_id, handler(params))
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi_language_server\server.py", line 549, in code_action
    jedi_script = jedi_utils.script(server.project, document)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi_language_server\jedi_utils.py", line 118, in script
    return Script(code=document.source, path=document.path, project=project)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\api\__init__.py", line 119, in __init__
    self._inference_state = InferenceState(
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\inference\__init__.py", line 87, in __init__
    environment = project.get_environment()
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\api\project.py", line 245, in get_environment
    self._environment = create_environment(self._environment_path, safe=False)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\api\environment.py", line 366, in create_environment
    return Environment(path, env_vars=env_vars)
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\api\environment.py", line 68, in __init__
    self._get_subprocess()
  File "c:\Users\jpele\.vscode\extensions\ms-python.python-2023.25.10292213\pythonFiles\lib\jedilsp\jedi\api\environment.py", line 79, in _get_subprocess
    raise InvalidPythonEnvironment(
jedi.api.environment.InvalidPythonEnvironment: Could not get version information for 'C:\\Program Files (x86)\\wapt\\Scripts\\python.exe': InternalError("The subprocess C:\\Program Files (x86)\\wapt\\Scripts\\python.exe has crashed (EOFError('Ran out of input'), stderr=).")

Ref issue from VS Code python Extension: https://github.com/microsoft/vscode-python/issues/21265

In python extension we started passing the path to selected python executable to Jedi. It seems like in some cases Jedi fails to get the version information with a strange error. Not sure if any recovery steps can be used. One enhancement I think could be: if the sys.executable path is the same as the environment path, then probably there is no need to run this to get more info.

[davidhalter]

Generally I don't think Jedi is doing something wrong here.

It seems like in some cases Jedi fails to get the version information with a strange error.

This is unfortunately not a strange error, but a fundamental issue with the executable. When started in a subprocess, the executable just dies.

The issue to me is very unclear here. It feels like a broken Python. Or something specific to Windows where it's not possible to start links as a subprocess or whatever. I have really no Windows specific knowledge.

Not sure if any recovery steps can be used.

There are currently no recovery steps, because this was never an issue as far as I know. I haven't had any reports (other than the bugs we fixed).

It would be possible that you guys catch this specific exception and just pass an InterpreterEnvironment as a fallback, which would mean that Jedi would run within the same Python that you start it. Unless it happens to a lot of your users I probably wouldn't do that, because that just hides the actual issues (which we do not understand yet). How many issues like that have you had?

One enhancement I think could be: if the sys.executable path is the same as the environment path, then probably there is no need to run this to get more info.

As I wrote above, that is already there if you want it, it's called InterpreterEnvironment.

[karthiknadig]

I think we should be able to do what you suggest as a fallback. Also, sorry for assuming this was an error in Jedi.

This is likely a suggestion to jedi-language-server package, as that is the one sitting between VS Code and Jedi. I will raise an issue there with details.

[davidhalter]

I'm not sure that's a good idea, just because we don't know yet why this is happening. How often does this happen to your users?

Also, sorry for assuming this was an error in Jedi.

No worries.

[karthiknadig]

This is the first time I have seen this being reported. This started happening after we started passing the path to environment dir or the python binary in the environment. This was part of fix to improve virtual environment experience with jedi.

[davidhalter]

I just feel like this is something that a user wants to know. So if you add a fallback, it might also generate github issues about sys.paths that are not found at the right places and at that point the fallback just bites you.

So I feel like this is really an issue of the underlying system of the original reporter. However a fallback could at least provide partial support for completions...