Closed traversaro closed 4 weeks ago
traversaro
commented
6 months ago I noticed that mamba info and conda info report as __archspec virtual package available __archspec=1=skylake, while micromamba reports __archspec=1=x86_64-v3, that perhaps could be the source behind the error. Probably this can be handled on the https://github.com/conda-forge/microarch-level-feedstock side, but I am not sure what is the best way forward on this.
hmaarrfk
commented
5 months ago I've found that my environments that require a __cuda package seem to fail. For example, I created https://github.com/conda-forge/staged-recipes/pull/26116 but the variant that is created with __cuda fails to solve for new environments.
mamba create --name t cupyifcudaavailable=*=cuda_* python=3.10will fail with
Could not solve for environment specs
The following package could not be installed
└─ cupyifcudaavailable * cuda_* is not installable because it requires
└─ __cuda, which is missing on the system.in fact, even after activating the environment, mamba will not recognize that one could "update" to the 100s level package to install .
conda with libmamba-solver seems to find the correct package.
hmaarrfk
commented
5 months ago Sorry for the noise on your issue. I think the problem with my issue is that mamba uses nvidia-smi to detect the cuda version: https://github.com/mamba-org/mamba/blob/46036e19171df59d69a0e381b490f55c467ed718/libmamba/src/core/virtual_packages.cpp#L72
whereas conda uses heuristics based on typical installation paths: https://github.com/anaconda/conda/blob/0dbf85e0546e0b0dc060c8265ec936591ccbe980/conda/common/cuda.py#L5
I was able to install nvidia-smi to resolve my issue with the __cuda virtual package.
I suspect there are other quirks with how virtual packages are detected for bleeding edge features.
Maybe conda + libmamba is enough to help you!
traversaro
commented
4 months ago Sorry @hmaarrfk, I had missed your comment. Yes, the problem are similar but different, they boil down to virtual packages to be implemented in different ways in conda and mamba (and also rattler/pixi), so these issues emerge. Indeed the long term solution is for users to use either conda or rattler/pixi, so we reduce the number of different virtual packages implementations from 3 to 2, but users tend to me lazy in change tools (especially when muscle memory of commands to type is involved).
The main problem here is that in the context of https://github.com/conda-forge/conda-forge.github.io/issues/1261 if you try to add archspec support as documented (as of May 2024) in https://conda-forge.org/docs/maintainer/knowledge_base/#microarch, this will result in packages that silently can't be installed at all with mamba, that may be an issue difficult to debug. Probably a possible strategy is to remove the x86_64-microarch-level dependency in unix and x86_64 and microarch_level == 1 builds, so that the downside in mamba will remain that you can't install AVX2 or higher builds, that may be more acceptable as downside.
traversaro
commented
1 month ago I just tested with mamba 2 and the problem is still present. As more packages are added to conda-forge that use archspec-optimized builds as described in https://conda-forge.org/docs/maintainer/knowledge_base/#microarch, all such builds (even the one with microarch_level==1) can't be installed with mamba, resulting in mamba silently installing old version of the packages that are not archspec-optimized.
For example, trying to install the latest version of the graph-tool package works fine on conda, but fails on mamba.
conda:
traversaro@IITBMP014LW012:~$ conda create -n graph-tool graph-tool==2.77.*
Channels:
- conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 24.3.0
latest version: 24.7.1
Please update conda by running
$ conda update -n base -c conda-forge conda
## Package Plan ##
environment location: /home/traversaro/miniforge3/envs/graph-tool
added / updated specs:
- graph-tool=2.77
The following packages will be downloaded:
[...]mamba:
traversaro@IITBMP014LW012:~$ mamba create -n graph-tool graph-tool==2.77.*
Looking for: ['graph-tool=2.77']
warning libmamba Could not parse mod/etag header
warning libmamba Could not parse mod/etag header
conda-forge/noarch 16.0MB @ 638.8kB/s 25.0s
conda-forge/linux-64 36.9MB @ 1.1MB/s 32.2s
Could not solve for environment specs
The following packages are incompatible
└─ graph-tool 2.77** is not installable because there are no viable options
├─ graph-tool 2.77 would require
│ └─ graph-tool-base [2.77 py310h9e900f9_300|2.77 py311haefbb00_300|2.77 py312h24de238_300|2.77 py38h8c1b0d4_300|2.77 py39hf24311c_300], which requires
│ └─ _x86_64-microarch-level >=3 but there are no viable options
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 broadwell, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 cannonlake, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 excavator, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 haswell, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 mic_knl, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 skylake, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 x86_64_v3, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 zen, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 zen2, which is missing on the system;
│ ├─ _x86_64-microarch-level 3 would require
│ │ └─ __archspec 1 zen3, which is missing on the system;
│ ├─ _x86_64-microarch-level 4 would require
│ │ └─ __archspec 1 cascadelake, which is missing on the system;
│ ├─ _x86_64-microarch-level 4 would require
│ │ └─ __archspec 1 icelake, which is missing on the system;
│ ├─ _x86_64-microarch-level 4 would require
│ │ └─ __archspec 1 skylake_avx512, which is missing on the system;
│ ├─ _x86_64-microarch-level 4 would require
│ │ └─ __archspec 1 x86_64_v4, which is missing on the system;
│ ├─ _x86_64-microarch-level 4 would require
│ │ └─ __archspec 1 zen4, which is missing on the system;
│ └─ _x86_64-microarch-level 4 would require
│ └─ __archspec 1 sapphirerapids, which is missing on the system;
├─ graph-tool 2.77 would require
│ └─ graph-tool-base [2.77 py310hc5c3ccb_100|2.77 py311h0951e8c_100|2.77 py312h7379947_100|2.77 py38hee84d54_100|2.77 py39h45f8f1b_100], which requires
│ └─ _x86_64-microarch-level >=1 but there are no viable options
│ ├─ _x86_64-microarch-level 1 would require
│ │ └─ __archspec 1 core2, which is missing on the system;
│ ├─ _x86_64-microarch-level 1 would require
│ │ └─ __archspec 1 k10, which is missing on the system;
│ ├─ _x86_64-microarch-level 1 would require
│ │ └─ __archspec 1 nocona, which is missing on the system;
│ ├─ _x86_64-microarch-level 1 would require
│ │ └─ __archspec 1 x86_64, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 bulldozer, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 ivybridge, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 nehalem, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 piledriver, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 sandybridge, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 steamroller, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 westmere, which is missing on the system;
│ ├─ _x86_64-microarch-level 2 would require
│ │ └─ __archspec 1 x86_64_v2, which is missing on the system;
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 3, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 4, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 4, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 4, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 4, which cannot be installed (as previously explained);
│ ├─ _x86_64-microarch-level 4, which cannot be installed (as previously explained);
│ └─ _x86_64-microarch-level 4, which cannot be installed (as previously explained);
└─ graph-tool 2.77 would require
└─ graph-tool-base [2.77 py310h6300a93_400|2.77 py311hf115aa5_400|2.77 py312h71012fb_400|2.77 py38h33ea7c3_400|2.77 py39heddc103_400], which requires
└─ _x86_64-microarch-level 4.* , which cannot be installed (as previously explained).By looking a bit more into this, the problems seems that mamba reports generic microarch as x86_64-v3 (see https://github.com/mamba-org/mamba/blob/83fdeae46572a05a1657395cccccbab74e8710d3/libmamba/src/core/virtual_packages.cpp#L177-L191), while conda force expects x86_64_v3, see https://github.com/conda-forge/microarch-level-feedstock/blob/1b446dbab3999c5e27f1e0b000231f18fa1b6df1/recipe/conda_build_config.yaml#L15 .
Probably it make sense to open a sister issue in https://github.com/conda-forge/microarch-level-feedstock to let the maintainers there be aware of this.
isuruf
commented
1 month ago Yeah we should fix it to report x86_64_v3 which is compliant with archspec.
traversaro
commented
1 month ago Yeah we should fix it to report
x86_64_v3which is compliant with archspec.
Thanks, done in https://github.com/mamba-org/mamba/pull/3404 .
Troubleshooting docs
Anaconda default channels
How did you install Mamba?
Mambaforge or latest Miniforge
Search tried in issue tracker
archspec
Latest version of Mamba
Tried in Conda?
I do not have this problem with Conda, just with Mamba
Describe your issue
If I try to run
mamba -v create -n test _x86_64-microarch-levelormicromamba -v create -n test _x86_64-microarch-level, the environment creation fails. If I tryconda create -n test _x86_64-microarch-levelor install the_x86_64-microarch-levelpackage in pixi, the installation in successful.Related to https://github.com/conda-forge/staged-recipes/pull/24306 and https://github.com/conda-forge/conda-forge.github.io/pull/2091 .
mamba info / micromamba info
Logs
environment.yml
~/.condarc