macro128 / pdm-conda

A PDM plugin to resolve/install/uninstall project dependencies with Conda
33 stars 1 forks source link

`pdm install` with `as-default-manager = false` needlessly attempts to fetch MAPPINGS_URL #15

Closed tysonclugg closed 11 months ago

tysonclugg commented 1 year ago

In an environment with restricted network access, I have a pyproject.toml like this:

[project]
name = "foo"
version = "1.0.0"
dependencies = [
    "psycopg[binary]>=3.1",
]
requires-python = ">=3.11"

[tool.pdm.conda]
dependencies = [
    "libpq",
]
as-default-manager = false

[tool.pdm]
plugins = [
    "pdm-conda",
]

This should install libpq via conda-forge, and psycopg[binary]>=3.1 via pypi.

However, when I try to install using pdm install -v, pdm-conda unnecessarily attempts to resolve the conda_name of the psycopg requirement and I get the following traceback as an attempt is made to download the conda to pypi MAPPINGS_URL (from github.com which is blocked):

Failed to load plugin conda=pdm_conda:main: can only concatenate tuple (not "list") to tuple
Traceback (most recent call last):
  File "/REDACTED-A/bin/pdm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/core.py", line 290, in main
    return Core().main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/core.py", line 208, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/core.py", line 203, in main
    self.handle(project, options)
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/core.py", line 157, in handle
    command.handle(project, options)
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/cli/commands/install.py", line 95, in handle
    actions.do_sync(
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/cli/actions.py", line 196, in do_sync
    requirements.extend(project.get_dependencies(group).values())
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/project/core.py", line 330, in get_dependencies
    result[req.identify()] = req
           ^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/pdm/models/requirements.py", line 131, in identify
    if not self.key:
           ^^^^^^^^
  File "/REDACTED-B/.pdm-plugins/lib/python3.11/site-packages/pdm_conda/models/requirements.py", line 269, in key
    return normalize_name(self.conda_name).lower() if self.conda_name else None
                                                      ^^^^^^^^^^^^^^^
  File "/REDACTED-B/.pdm-plugins/lib/python3.11/site-packages/pdm_conda/models/requirements.py", line 264, in conda_name
    self._conda_name = pypi_to_conda(strip_extras(name)[0]) if name else None
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-B/.pdm-plugins/lib/python3.11/site-packages/pdm_conda/mapping.py", line 100, in pypi_to_conda
    return _requirement_map(requirement, get_pypi_mapping()).lower()
                                         ^^^^^^^^^^^^^^^^^^
  File "/REDACTED-B/.pdm-plugins/lib/python3.11/site-packages/pdm_conda/mapping.py", line 75, in get_pypi_mapping
    mapping = download_mapping(Path(str(download_dir)))
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-B/.pdm-plugins/lib/python3.11/site-packages/pdm_conda/mapping.py", line 52, in download_mapping
    response = requests.get(MAPPINGS_URL, stream=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/REDACTED-A/lib/python3.11/site-packages/requests/adapters.py", line 501, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

You should be able to replicate this issue (but with a subtly different traceback) by adding a bogus entry for github.com in /etc/hosts and flushing your DNS cache.

This issue also raises a companion bug, there should be a way to override the MAPPINGS_URL to a privately mirrored copy if necessary.

macro128 commented 11 months ago

Sure thing, I'll add it on the next version!

macro128 commented 11 months ago

Added since v0.12.0