Failed to download compressed mapping from GitHub, even if I can `wget` the file.

traversaro commented 2 months ago

Checks

[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of pixi, using pixi --version: 0.27.1

Reproducible example

I am on a HPC system (of which I do not know all the details, so this is definitely not reproducible) and running a pixi command fails with:

[straversaro@fnode01 jaxsim]$ pixi run -e test-gpu test
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
  × failed to download pypi mapping from https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json location
  ├─▶ Middleware error: File still doesn't exist
  ├─▶ File still doesn't exist
  ╰─▶ No such file or directory (os error 2)
~~~

Interestingly, exactly the same file can be downloaded without problem with
~~~
[straversaro@fnode01 jaxsim]$ wget  https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json
--2024-08-22 10:51:43--  https://raw.githubusercontent.com/prefix-dev/parselmouth/main/files/compressed_mapping.json
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.109.133, 185.199.110.133, 185.199.111.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.109.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 714886 (698K) [text/plain]
Saving to: ‘compressed_mapping.json’

compressed_mapping.json                     100%[===========================================================================================>] 698.13K  --.-KB/s    in 0.06s   

2024-08-22 10:51:44 (11.2 MB/s) - ‘compressed_mapping.json’ saved [714886/714886]

Issue description

I found a bunch of similar (but apparently not directly related) issues, so I tought it was more clear to open a new issue. Do you have any idea on how I could debug the problem more?

Expected behavior

That pixi run worked without any download error.

ruben-arts commented 2 months ago

Did you try with pixi run --tls-no-verify I'm curious if that would fix it.

traversaro commented 2 months ago

It still does not work (and now the message slightly changed, it does not print the url of the file to download anymore):

[straversaro@fe01 jaxsim]$ pixi run echo
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
  ⠒ gpugroup:linux-64    [00:00:04] loading repodata
  ⠒ cpugroup:linux-64    [00:00:04] applying JLAP patches
  ⠒ cpugroup:osx-64      [00:00:04] loading repodata
  × failed to download pypi name mapping
  ├─▶ Middleware error: File still doesn't exist
  ├─▶ File still doesn't exist
  ╰─▶ No such file or directory (os error 2)

[straversaro@fe01 jaxsim]$ pixi run --tls-no-verify echo
 WARN The feature 'style' is defined but not used in any environment
 WARN The feature 'testing' is defined but not used in any environment
 WARN The feature 'viz' is defined but not used in any environment
 WARN The feature 'all' is defined but not used in any environment
 WARN TLS verification is disabled. This is insecure and should only be used for testing or internal networks.
  ⠤ cpugroup:linux-64    [00:00:04] applying JLAP patches
  ⠤ gpugroup:linux-64    [00:00:04] loading repodata
  ⠤ cpugroup:osx-64      [00:00:04] loading repodata
  × failed to download pypi name mapping
  ├─▶ Middleware error: File still doesn't exist
  ├─▶ File still doesn't exist
  ╰─▶ No such file or directory (os error 2)

ctcjab commented 2 months ago

When I add config like the following to my pyproject.toml that the docs suggest...

[tool.pixi.pypi-dependencies]
foo = { path = ".", editable = true }

...and then try to pixi install, I get the error:

  × failed to download pypi name mapping
  ├─▶ error sending request for url (https://conda-mapping.prefix.dev/hash-v0/2c9874a8f76f8edabb9213351be27d8cc8eed2a08428b230e2b87cd00b9a06d8)
  ├─▶ client error (Connect)
  ├─▶ dns error: failed to lookup address information: Try again
  ╰─▶ failed to lookup address information: Try again

Looks similar but not exactly the same as the error reported in this issue -- should I open a separate issue?

nichmor commented 2 months ago

When I add config like the following to my pyproject.toml that the docs suggest...
[tool.pixi.pypi-dependencies]
foo = { path = ".", editable = true }
...and then try to pixi install, I get the error:
  × failed to download pypi name mapping
  ├─▶ error sending request for url (https://conda-mapping.prefix.dev/hash-v0/2c9874a8f76f8edabb9213351be27d8cc8eed2a08428b230e2b87cd00b9a06d8)
  ├─▶ client error (Connect)
  ├─▶ dns error: failed to lookup address information: Try again
  ╰─▶ failed to lookup address information: Try again
Looks similar but not exactly the same as the error reported in this issue -- should I open a separate issue?

Hey! could you please share your pixi.toml or pyproject.toml configuration?

ctcjab commented 2 months ago

Sure:

[project]
name = "foo"
description = "foo"
version = "0.0.1"

[tool.pixi.project]
channels = [
  "https://artifactory.chicagotrading.com/artifactory/api/conda/ctc-curated-condaforge-main",
  ...
]
platforms = ["linux-64"]

[tool.pixi.dependencies]
# 3rd-party
matplotlib = "*"
pandas = "*"
# 1st-party
...

# The commented out config below is suggested by
# https://pixi.sh/latest/tutorials/python/#whats-in-the-pyprojecttoml:~:text=The%20pixi_py%20package%20itself%20is%20added%20as%20an%20editable%20dependency
# for a good local development workflow, but caused the following error when I tried it:
# error sending request for url (https://conda-mapping.prefix.dev/hash-v0/115b796fddc846bee6f47e3c57d04d12fa93a47a7a8ef639cefdc05203c1bf00)
# ├─▶ client error (Connect)
# ├─▶ dns error: failed to lookup address information: Try again
#
# [tool.pixi.pypi-dependencies]
# foo = { path = ".", editable = true }
#
# Install this package in the test env too so that we can remove "src" from pytest pythonpath config below:
# [tool.pixi.feature.test.pypi-dependencies]
# foo = { path = ".", editable = true }

[tool.pixi.feature.test.dependencies]
pytest = "*"

[tool.pixi.feature.test.tasks]
test = "pytest"

[tool.pixi.environments]
default = { solve-group = "default" }
test = { features = ["test"], solve-group = "default" }

[tool.pytest.ini_options]
pythonpath = ["src"]
testpaths = ["tests"]
addopts = [
  "-vv",
]

ricrogz commented 3 weeks ago

Did you find what was going on? I think I'm hitting the same issue on Ubuntu 22.04, and I can't but suspect it is related to authentication and access to gnome-keyring.

ctcjab commented 3 weeks ago

I did not ever find out what was going on, and ended up switching away from pixi for the use case where I was hitting this issue, so I haven't put more time into it since then.

ricrogz commented 1 week ago

Ok, I found out what was happening on my case: I was using the pixi executable from the Releases section here in github, inside of a Rocky8 docker container. That one is built on top of musl. And musl seems to have a known issue with dns resolving: https://stackoverflow.com/questions/65181012/does-alpine-have-known-dns-issue-within-kubernetes#65593511. The stackoverflow post is old (2021!), but it seems the issue is still there in some degree, as I could work around it by using one of the suggested workarounds: direct edition of /etc/host to hardcode some FQDN to their known IPs.

It might be a good idea to add a glibc based build to the release assets.

prefix-dev / pixi