prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
3.44k stars 197 forks source link

Failure to solve conda requirements with custom conda repository #2371

Open Krande opened 1 month ago

Krande commented 1 month ago

Checks

Reproducible example

Using this minimal pixi.toml file with pixi v0.34.0

[project]
name = "pixi-tests"
version = "1.0.0"
description = "A pixi project for testing"
channels = ["https://software.repos.intel.com/python/conda", "conda-forge"]
platforms = ["win-64"]

[dependencies]
ifx_win-64 = "*"

[tasks]
info = {cmd=[ "where", "ifx"]}

I also made a test repo: https://github.com/Krande/pixi_testing

Issue description

Fails to find the package "ifx_win-64".

(base) C:\Work\code\code-aster-src>pixi run info
  × failed to solve the conda requirements of 'default' 'win-64'
  ╰─▶ Cannot solve the request because of: No candidates were found for ifx_win-64 *.

Expected behavior

When I try using mamba it finds the package.

(base) C:\Work\code\code-aster-src>mamba create -n mambatest -c https://software.repos.intel.com/python/conda -c conda-forge ifx_win-64

Looking for: ['ifx_win-64']

https://software.repos.intel.com/python/conda/no..            No change
https://software.repos.intel.com/python/conda/wi..            No change
conda-forge/noarch                                  17.0MB @  29.5MB/s  0.6s
conda-forge/win-64                                  25.2MB @  32.3MB/s  0.8s
Transaction                                                                                                                                                                      
  Prefix: C:\Work\miniforge3\envs\mambatest

  Updating specs:

   - ifx_win-64

  Package                    Version  Build           Channel                                     Size
--------------------------------------------------------------------------------------------------------
  Install:
--------------------------------------------------------------------------------------------------------

  + compiler_shared         2025.0.0  intel_1162      software.repos.intel.com/python/conda      195MB
  + intel-cmplr-lib-rt      2025.0.0  intel_1162      software.repos.intel.com/python/conda       16MB
  + intel-cmplr-lic-rt      2025.0.0  intel_1162      software.repos.intel.com/python/conda       47kB
  + tcm                        1.2.0  intel_583       software.repos.intel.com/python/conda      332kB
  + umf                        0.9.0  intel_584       software.repos.intel.com/python/conda       63kB
  + intel-cmplr-lib-ur      2025.0.0  intel_1162      software.repos.intel.com/python/conda        1MB
  + intel-openmp            2025.0.0  intel_1162      software.repos.intel.com/python/conda        4MB
  + intel-fortran-rt        2025.0.0  intel_1162      software.repos.intel.com/python/conda        3MB
  + vswhere                    3.1.7  h57928b3_0      conda-forge                                219kB
  + ucrt                10.0.22621.0  h57928b3_1      conda-forge                               Cached
  + vs2022_win-64        19.41.34120  h72b6792_22     conda-forge                                 20kB
  + vc14_runtime         14.40.33810  hcc2c482_22     conda-forge                               Cached
  + vs_win-64                2022.11  h72b6792_22     conda-forge                                 18kB
  + vc                          14.3  h8a93ad2_22     conda-forge                               Cached
  + tbb                     2022.0.0  vc14_intel_396  software.repos.intel.com/python/conda      233kB
  + intel-opencl-rt         2025.0.0  intel_1162      software.repos.intel.com/python/conda      100MB
  + ifx_impl_win-64         2025.0.0  intel_1162      software.repos.intel.com/python/conda       29MB
  + ifx_win-64              2025.0.0  intel_1162      software.repos.intel.com/python/conda       18kB

  Summary:

  Install: 18 packages

  Total download: 348MB

--------------------------------------------------------------------------------------------------------

Confirm changes: [Y/n]

Is this is a bug in pixi, or am I doing something wrong (which is completely plausible)?

Best Regards Kristoffer

ruben-arts commented 1 month ago

I can't find a way to install these as well, I do find that I can install A version of intel-opencl-rt but not in the same version as mamba. Asked @wolfv to take a look as he probally understands the channel better.

wolfv commented 1 month ago

Hi @Krande - I think this might be related to weird unicode characters in the repodata. I see a bunch of "common_cmplr_lib_rt \u003E=2024.2.0", in the repodata when visiting https://software.repos.intel.com/python/conda/win-64/repodata.json

I don't really think that rattler deals well with that. I am wondering why / how conda mamba work (do they decode this data or ignore it? ...

wolfv commented 1 month ago

Hmm, although on closer inspection, things seem well decoded when running pixi search on that channel / subdir. E.g. pixi search intelpython3_full -c https://software.repos.intel.com/python/conda --platform=win-64 shows good results.

I'll have to dig deeper - later today!

wolfv commented 1 month ago

OK, so it's still unclear wether the unicode things are a problem because pixi takes the bz2 repodata. That repodata seems incomplete, but also not affected by the Unicode issues.

You can work around it by disabling bz2 in pixi config (using e.g. pixi config edit --global).

[repodata-config]
# disable fetching of jlap, bz2 or zstd repodata files.
# This should only be used for specific old versions of artifactory and other non-compliant
# servers.
disable-bzip2 = true # don't try to download repodata.json.bz2
Krande commented 1 month ago

Thanks @wolfv! I can confirm that your suggested workaround works for me :)