Test: Selecting optional dependencies from `pyproject.toml` while using Create Environment command

[karthiknadig]

Refs: https://github.com/microsoft/vscode-python/issues/20278

• [x] windows @minsa110
• [x] linux @alexr00
• [x] macOS @paulacamargo25

Complexity: 2

Create Issue

---

Requirements

1. Latest pre-release of the python extension.
2. A global Python 3.7 or greater.

Verification

1. Create a folder or open a project containing pyproject.toml files.
2. Run the Python: Create Environment command.
3. Select Venv option
4. Select a global python.
5. It should show you optional dependencies found in the project. For example, if your pyproject.toml file has these entries [project.optional-dependencies], you should see test,lint,cov:

   [project.optional-dependencies]
   test = ["pytest"]
   lint = [ "black", "flake8"]
   cov = ["pytest < 5.0.0", "pytest-cov[all]"]

6. You should be able select one, multiple or none.
7. Look at the logs to ensure all the requirements were installed.

NOTE: If you want to retest with different combination of optional deps please delete the .venv manually.

[alexr00]

I also had to do the following: Upgrade pip /usr/local/bin/python3 -m pip install --upgrade pip Install python venv apt-get install python3-venv

But I've still reached an error that I'm not sure how to proceed on:

Running Env creation script:  [
  '/usr/bin/python3',
  '/home/vscode/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/create_venv.py',
  '--git-ignore',
  '--toml',
  '/workspaces/vscode-remote-try-python/pyproject.toml'
]
Running: /usr/bin/python3 -m venv .venv

> ./.venv/bin/python -I ~/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/get_output_via_markers.py ~/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/interpreterInfo.py
CREATED_VENV:/workspaces/vscode-remote-try-python/.venv/bin/python

Creating: /workspaces/vscode-remote-try-python/.venv/.gitignore
Running: /workspaces/vscode-remote-try-python/.venv/bin/python -m pip install --upgrade pip

Collecting pip

  Using cached pip-22.3.1-py3-none-any.whl (2.1 MB)

Installing collected packages: pip

Successfully installed pip-22.3.1

VENV_INSTALLING_PYPROJECT: /workspaces/vscode-remote-try-python/pyproject.toml
Running: /workspaces/vscode-remote-try-python/.venv/bin/python -m pip install -e .

Obtaining file:///workspaces/vscode-remote-try-python

  Installing build dependencies: started

  Installing build dependencies: finished with status 'error'

  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      Collecting hatchling
        Using cached hatchling-1.12.2-py3-none-any.whl (79 kB)
      Collecting editables>=0.3
        Using cached editables-0.3-py3-none-any.whl (4.7 kB)
      Collecting packaging>=21.3
        Using cached packaging-23.0-py3-none-any.whl (42 kB)
      Collecting pluggy>=1.0.0
        Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
      Collecting pathspec>=0.10.1
        Using cached pathspec-0.10.3-py3-none-any.whl (29 kB)
      Collecting tomli>=1.2.2
        Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
      Installing collected packages: tomli, pluggy, pathspec, packaging, editables, hatchling
      ERROR: Cannot set --home and --prefix together
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.

error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Traceback (most recent call last):

  File "/home/vscode/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/create_venv.py", line 75, in run_process
    subprocess.run(args, cwd=os.getcwd(), check=True)
  File "/usr/lib/python3.9/subprocess.py", line 528, in run

raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/workspaces/vscode-remote-try-python/.venv/bin/python', '-m', 'pip', 'install', '-e', '.']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vscode/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/create_venv.py", line 167, in <module>

main(sys.argv[1:])
  File "/home/vscode/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/create_venv.py", line 163, in main

    install_toml(venv_path, args.extras)
  File "/home/vscode/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/create_venv.py", line 106, in install_toml
    run_process(
  File "/home/vscode/.vscode-server-insiders/extensions/ms-python.python-2023.1.10232135/pythonFiles/create_venv.py", line 77, in run_process

    raise VenvError(error_message)
__main__.VenvError: CREATE_VENV.PIP_FAILED_INSTALL_PYPROJECT

[ERROR 2023-0-24 13:25:5.966]: Error while running venv creation script:  CREATE_VENV.PIP_FAILED_INSTALL_PYPROJECT
[ERROR 2023-0-24 13:25:5.967]: CREATE_VENV.PIP_FAILED_INSTALL_PYPROJECT

[karthiknadig]

@alexr00 If you got the UI to pick the items to select then you are good for this TPI.

For this error, can you share the toml file and python version and OS version.

[minsa110]

I got the UI to pick the optional dependencies ✅:

But the operation failed:

Here are my logs (seems same as Alex's):

Experiment 'pythonSurveyNotification' is active
Experiment 'pythonPromptNewToolsExt' is active
LSP Notebooks experiment is enabled
LSP Notebooks interactive window support is enabled
Python interpreter path: ~\AppData\Local\Microsoft\WindowsApps\python3.10.exe
Starting Pylance language server.
> conda info --json
> ~\AppData\Local\Microsoft\WindowsApps\python3.10.exe -I ~\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\get_output_via_markers.py ~\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\interpreterInfo.py
> "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\python3.10.exe" -I ~\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\get_output_via_markers.py ~\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\interpreterInfo.py
Running Env creation script:  [
  'C:\\Users\\somin\\AppData\\Local\\Microsoft\\WindowsApps\\python3.10.exe',
  'c:\\Users\\somin\\.vscode-insiders\\extensions\\ms-python.python-2023.1.10241007\\pythonFiles\\create_venv.py',
  '--git-ignore',
  '--toml',
  '"c:/Users/somin/OneDrive - Microsoft/Documents/Test Project 3/pyproject.toml"',
  '--extras',
  'dev'
]
Running: C:\Users\somin\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\python.exe -m venv .venv

> "~\OneDrive - Microsoft\Documents\Test Project 3\.venv\Scripts\python.exe" -I ~\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\get_output_via_markers.py ~\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\interpreterInfo.py
CREATED_VENV:c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\Scripts\python.exe

Creating: c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\.gitignore

Running: c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\Scripts\python.exe -m pip install --upgrade pip

Requirement already satisfied: pip in c:\users\somin\onedrive - microsoft\documents\test project 3\.venv\lib\site-packages (22.3.1)

VENV_INSTALLING_PYPROJECT: "c:/Users/somin/OneDrive - Microsoft/Documents/Test Project 3/pyproject.toml"
Running: c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\Scripts\python.exe -m pip install -e .[dev]

Obtaining file:///C:/Users/somin/OneDrive%20-%20Microsoft/Documents/Test%20Project%203

  Installing build dependencies: started

  Installing build dependencies: finished with status 'done'

  Checking if build backend supports build_editable: started

  Checking if build backend supports build_editable: finished with status 'done'

  Getting requirements to build editable: started

  Getting requirements to build editable: finished with status 'error'

  error: subprocess-exited-with-error

  ×
 Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      Traceback (most recent call last):
        File "c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 351, in <module>
          main()
        File "c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 333, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "c:\Users\somin\OneDrive - Microsoft\Documents\Test Project 3\.venv\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 132, in get_requires_for_build_editable
          return hook(config_settings)
        File "C:\Users\somin\AppData\Local\Temp\pip-build-env-m_2k01rz\overlay\Lib\site-packages\flit_core\buildapi.py", line 31, in get_requires_for_build_wheel
          module = Module(info.module, Path.cwd())
        File "C:\Users\somin\AppData\Local\Temp\pip-build-env-m_2k01rz\overlay\Lib\site-packages\flit_core\common.py", line 59, in __init__
          raise ValueError("No file/folder found for module {}".format(name))
      ValueError: No file/folder found for module titanic_ml
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.

error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 
1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Traceback (most recent call last):
  File "c:\Users\somin\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\create_venv.py", line 75, in run_process

    subprocess.run(args, cwd=os.getcwd(), check=True)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\subprocess.py", line 526, in run

    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['c:\\Users\\somin\\OneDrive - Microsoft\\Documents\\Test Project 3\\.venv\\Scripts\\python.exe', '-m', 'pip', 'install', '-e', '.[dev]']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\somin\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\create_venv.py", line 167, in <module>

    main(sys.argv[1:])
  File "c:\Users\somin\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\create_venv.py", line 163, in main

    install_toml(venv_path, args.extras)
  File "c:\Users\somin\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\create_venv.py", line 106, in install_toml

    run_process(
  File "c:\Users\somin\.vscode-insiders\extensions\ms-python.python-2023.1.10241007\pythonFiles\create_venv.py", line 77, in run_process

    raise VenvError(error_message)
__main__.VenvError: CREATE_VENV.PIP_FAILED_INSTALL_PYPROJECT

[ERROR 2023-0-24 10:56:33.479]: Error while running venv creation script:  CREATE_VENV.PIP_FAILED_INSTALL_PYPROJECT
[ERROR 2023-0-24 10:56:33.480]: CREATE_VENV.PIP_FAILED_INSTALL_PYPROJECT

[paulacamargo25]

I got the UI to pick the optional dependencies:

I tried choosing all the options, only one and none. In all cases the creation of the environment was correct.

Great job :)

[karthiknadig]

@minsa110 In your case as well, the issue is not with the UI but the build tool that is being used to install the project. can you share the project or the TOML file.

@minsa110 @alexr00 Do open an issue for your cases. This TPI was to test if the UI provides the right items from the toml. And if the command that we run is ensuring that the selections are passed through. As far as I can tell from the above logs, it seems to be passing through correctly. But, the failure is further down the line where the dependency installation is failing. we need to investigate that separately.

What to look for as success for this TPI, in the logs you should see something like this

• none case: -m pip install -e . ( -e with just a dot)
• one case (say dev) : -m pip install -e .[dev]
• many case (say dev, test, cov): -m pip install -e .[dev,test,cov] (order of dev, test, cov does not matter).