prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
2.92k stars 161 forks source link

Failed to download compressed mapping from GitHub, even if I can `wget` the file. #1882

Open traversaro opened 3 weeks ago

traversaro commented 3 weeks ago

Checks

Reproducible example

I am on a HPC system (of which I do not know all the details, so this is definitely not reproducible) and running a pixi command fails with:

[straversaro@fnode01 jaxsim]$ pixi run -e test-gpu test
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
  × failed to download pypi mapping from https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json location
  ├─▶ Middleware error: File still doesn't exist
  ├─▶ File still doesn't exist
  ╰─▶ No such file or directory (os error 2)
~~~

Interestingly, exactly the same file can be downloaded without problem with
~~~
[straversaro@fnode01 jaxsim]$ wget  https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json
--2024-08-22 10:51:43--  https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.110.133, 185.199.111.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 714886 (698K) [text/plain]
Saving to: ‘compressed_mapping.json’

compressed_mapping.json                     100%[===========================================================================================>] 698.13K  --.-KB/s    in 0.06s   

2024-08-22 10:51:44 (11.2 MB/s) - ‘compressed_mapping.json’ saved [714886/714886]

Issue description

I found a bunch of similar (but apparently not directly related) issues, so I tought it was more clear to open a new issue. Do you have any idea on how I could debug the problem more?

Expected behavior

That pixi run worked without any download error.

ruben-arts commented 3 weeks ago

Did you try with pixi run --tls-no-verify I'm curious if that would fix it.

traversaro commented 3 weeks ago

It still does not work (and now the message slightly changed, it does not print the url of the file to download anymore):

[straversaro@fe01 jaxsim]$ pixi run echo
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
  ⠒ gpugroup:linux-64    [00:00:04] loading repodata
  ⠒ cpugroup:linux-64    [00:00:04] applying JLAP patches
  ⠒ cpugroup:osx-64      [00:00:04] loading repodata
  × failed to download pypi name mapping
  ├─▶ Middleware error: File still doesn't exist
  ├─▶ File still doesn't exist
  ╰─▶ No such file or directory (os error 2)
[straversaro@fe01 jaxsim]$ pixi run --tls-no-verify echo
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
 WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
  ⠤ cpugroup:linux-64    [00:00:04] applying JLAP patches
  ⠤ gpugroup:linux-64    [00:00:04] loading repodata
  ⠤ cpugroup:osx-64      [00:00:04] loading repodata
  × failed to download pypi name mapping
  ├─▶ Middleware error: File still doesn't exist
  ├─▶ File still doesn't exist
  ╰─▶ No such file or directory (os error 2)
ctcjab commented 2 weeks ago

When I add config like the following to my pyproject.toml that the docs suggest...

[tool.pixi.pypi-dependencies]
foo = { path = ".", editable = true }

...and then try to pixi install, I get the error:

  × failed to download pypi name mapping
  ├─▶ error sending request for url (https://conda-mapping.prefix.dev/hash-v0/2c9874a8f76f8edabb9213351be27d8cc8eed2a08428b230e2b87cd00b9a06d8)
  ├─▶ client error (Connect)
  ├─▶ dns error: failed to lookup address information: Try again
  ╰─▶ failed to lookup address information: Try again

Looks similar but not exactly the same as the error reported in this issue -- should I open a separate issue?

nichmor commented 2 weeks ago

When I add config like the following to my pyproject.toml that the docs suggest...

[tool.pixi.pypi-dependencies]
foo = { path = ".", editable = true }

...and then try to pixi install, I get the error:

  × failed to download pypi name mapping
  ├─▶ error sending request for url (https://conda-mapping.prefix.dev/hash-v0/2c9874a8f76f8edabb9213351be27d8cc8eed2a08428b230e2b87cd00b9a06d8)
  ├─▶ client error (Connect)
  ├─▶ dns error: failed to lookup address information: Try again
  ╰─▶ failed to lookup address information: Try again

Looks similar but not exactly the same as the error reported in this issue -- should I open a separate issue?

Hey! could you please share your pixi.toml or pyproject.toml configuration?

ctcjab commented 2 weeks ago

Sure:

[project]
name = "foo"
description = "foo"
version = "0.0.1"

[tool.pixi.project]
channels = [
  "https://artifactory.chicagotrading.com/artifactory/api/conda/ctc-curated-condaforge-main",
  ...
]
platforms = ["linux-64"]

[tool.pixi.dependencies]
# 3rd-party
matplotlib = "*"
pandas = "*"
# 1st-party
...

# The commented out config below is suggested by
# https://pixi.sh/latest/tutorials/python/#whats-in-the-pyprojecttoml:~:text=The%20pixi_py%20package%20itself%20is%20added%20as%20an%20editable%20dependency
# for a good local development workflow, but caused the following error when I tried it:
# error sending request for url (https://conda-mapping.prefix.dev/hash-v0/115b796fddc846bee6f47e3c57d04d12fa93a47a7a8ef639cefdc05203c1bf00)
# ├─▶ client error (Connect)
# ├─▶ dns error: failed to lookup address information: Try again
#
# [tool.pixi.pypi-dependencies]
# foo = { path = ".", editable = true }
#
# Install this package in the test env too so that we can remove "src" from pytest pythonpath config below:
# [tool.pixi.feature.test.pypi-dependencies]
# foo = { path = ".", editable = true }

[tool.pixi.feature.test.dependencies]
pytest = "*"

[tool.pixi.feature.test.tasks]
test = "pytest"

[tool.pixi.environments]
default = { solve-group = "default" }
test = { features = ["test"], solve-group = "default" }

[tool.pytest.ini_options]
pythonpath = ["src"]
testpaths = ["tests"]
addopts = [
  "-vv",
]