EklavyaFCB
commented
2 years ago I have the same error !
Open arturdaraujo opened 2 years ago
EklavyaFCB
commented
2 years ago I have the same error !
jaimergp
commented
2 years ago Thank you so much for this report! I'll look into it.
davidandreoletti
commented
2 years ago @jaimergp Is this issue still active on your radar ?
jaimergp
commented
2 years ago I couldn't reproduce back then but I am taking another look this week, thanks for the reminder @davidandreoletti!
jaimergp
commented
2 years ago Still can't reproduce, here is what I did:
base environment after all these months.cf_python39 with the command conda install -c conda-forge -c defaults --strict-channel-priority python=3.9 vaex=4 jupyterlab (pretty large)Set libmamba as default: conda config --set experimental_solver libmamba
Deactivated base, activated cf_python39 and ran conda install scipy, to make sure the solver worked. It did.
conda clean --all -v --dry-run to see what would happen. You can see that all conda-libmamba-solver tarballs are removed, but one of their corresponding extracted directories was saved (the installed version, 22.6).conda clean --all -v again, but without the dry run. This WILL remove the packages, but should leave a working installation. It did. I can still use the libmamba solver.conda clean is designed to not remove linked packages, unless you use --force-pkgs-dirs, which has this nice disclaimer:
-f, --force-pkgs-dirs
Remove *all* writable package caches. This option is not included with the --all flag. WARNING: This will break environments with packages installed using symlinks back to the package cache.Why? conda installs work, mainly, by hardlinking the extracted files in the cache to the corresponding environment location. conda clean checks how many (hard)links a certain inode in the cache has. If it's greater than one, it must mean that is installed in an environment, so it skips it. You can see it here:
This exception is then ignored:
That said, I do see something weird in your setup. You have initialized cf_python39 in your shell, treating it like it was your base environment. But it's not. See these paths:
base environment : C:\Users\artur\miniconda3\envs\cf_python39 (writable)
conda av data dir : C:\Users\artur\miniconda3\envs\cf_python39\etc\conda
[...]
package cache : C:\Users\artur\miniconda3\envs\cf_python39\pkgs
C:\Users\artur\.conda\pkgs
C:\Users\artur\AppData\Local\conda\conda\pkgsYour base environment should be C:\Users\artur\miniconda3, not C:\Users\artur\miniconda3\envs\cf_python39. Same for the package cache; it should be C:\Users\artur\miniconda3\pkgs.
This kind of nested conda installations are not really supported, since you can run into these edge cases where it's not clear which environment is your base, functionally speaking. Although the link checks are done at the filesystem level and there's no mention of the active environment anywhere in the code, you might be running into a situation where you are executing a different conda than expected.
Let me know if this helped!
edit - read below for a better explanation - this is not your fault :)
jaimergp
commented
2 years ago For more detail, I did reproduce what I think it's happening in your case.
conda on cf_python39cf_python39's conda is now in PATH before base's conda.conda install from cf_python39, it will try to find conda-libmamba-solver in cf_python39, but it's not there, hence the error. conda-libmamba-solver in cf_python39, then it works again because it's importable in that environmnet.My logs:
(cf_python39) D:\Users\jrodriguez>conda install conda
***
NOTE: You are using the EXPERIMENTAL libmamba solver integration.
If something is not working as expected, please:
1. Go to https://github.com/conda/conda/issues/new/choose
2. Choose the "Libmamba Solver Feedback (Experimental Feature)" option
3. Attach the log file found in the following path:
D:\Users\jrodriguez\Miniconda3\pkgs\.logs\20220822-093739-704428.log
Thank you for your help!
***
pkgs/main/noarch No change
pkgs/main/win-64 No change
pkgs/r/win-64 No change
pkgs/msys2/noarch No change
pkgs/msys2/win-64 No change
pkgs/r/noarch No change
Collect all metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: D:\Users\jrodriguez\Miniconda3\envs\cf_python39
added / updated specs:
- conda
The following packages will be downloaded:
package | build
---------------------------|-----------------
conda-package-handling-1.8.1| py39h8cc25b3_0 729 KB
menuinst-1.4.18 | py39h59b6b97_0 96 KB
pycosat-0.6.3 | py39h2bbff1b_0 75 KB
ruamel_yaml-0.15.100 | py39h2bbff1b_0 273 KB
tqdm-4.64.0 | py39haa95532_0 155 KB
------------------------------------------------------------
Total: 1.3 MB
The following NEW packages will be INSTALLED:
conda pkgs/main/win-64::conda-4.13.0-py39haa95532_0
conda-package-han~ pkgs/main/win-64::conda-package-handling-1.8.1-py39h8cc25b3_0
menuinst pkgs/main/win-64::menuinst-1.4.18-py39h59b6b97_0
pycosat pkgs/main/win-64::pycosat-0.6.3-py39h2bbff1b_0
ruamel_yaml pkgs/main/win-64::ruamel_yaml-0.15.100-py39h2bbff1b_0
tqdm pkgs/main/win-64::tqdm-4.64.0-py39haa95532_0
Proceed ([y]/n)?
Downloading and Extracting Packages
ruamel_yaml-0.15.100 | 273 KB | ################################################################################################################################################################################################################################################################################# | 100%
tqdm-4.64.0 | 155 KB | ################################################################################################################################################################################################################################################################################# | 100%
pycosat-0.6.3 | 75 KB | ################################################################################################################################################################################################################################################################################# | 100%
conda-package-handli | 729 KB | ################################################################################################################################################################################################################################################################################# | 100%
menuinst-1.4.18 | 96 KB | ################################################################################################################################################################################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Retrieving notices: ...working... done
(cf_python39) D:\Users\jrodriguez>conda info
active environment : base
active env location : D:\Users\jrodriguez\Miniconda3\envs\cf_python39
shell level : 2
user config file : D:\Users\jrodriguez\.condarc
populated config files : D:\Users\jrodriguez\.condarc
conda version : 4.13.0
conda-build version : not installed
python version : 3.9.13.final.0
virtual packages : __win=0=0
__archspec=1=x86_64
base environment : D:\Users\jrodriguez\Miniconda3\envs\cf_python39 (writable) # SEE HERE!!! Same situation now :)
conda av data dir : D:\Users\jrodriguez\Miniconda3\envs\cf_python39\etc\conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/win-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/win-64
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-64
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : D:\Users\jrodriguez\Miniconda3\envs\cf_python39\pkgs # SEE HERE!!! Same situation now :)
D:\Users\jrodriguez\.conda\pkgs
D:\Users\jrodriguez\AppData\Local\conda\conda\pkgs
envs directories : D:\Users\jrodriguez\Miniconda3\envs\cf_python39\envs
D:\Users\jrodriguez\.conda\envs
D:\Users\jrodriguez\AppData\Local\conda\conda\envs
platform : win-64
user-agent : conda/4.13.0 requests/2.28.1 CPython/3.9.13 Windows/10 Windows/10.0.14393 solver/libmamba
administrator : False
netrc file : None
offline mode : False
(cf_python39) D:\Users\jrodriguez>conda install numpy
CondaImportError: You have chosen a non-default solver backend (libmamba) but it could not be imported:
ModuleNotFoundError: No module named 'conda_libmamba_solver'
Try (re)installing conda-libmamba-solver.So, all in all, I think it's an issue with the way activation works on Windows CMD consoles, where the conda command is provided by a .bat script that can get in PATH. On Unix, it's a shell function that is not usually overwritten by activation, because it comes from the initialization code.
This does not happen on Windows Powershell! I think because it also uses shell functions.
TLDR: This is the unfortunate but expected behaviour from Windows CMD activation logic. It has nothing to do with conda-libmamba-solver itself. We could rename the issue to "If conda is installed in a non-base environment, Windows CMD will treat that environment it as base once active", or similar.
qtdaniel
commented
1 year ago I don't think this is unique to Windows. I get similar behaviour on Ubuntu...
[username@hostname ~]$ conda config --env --show root_prefix 2> /dev/null
root_prefix: /opt/miniconda/miniconda-latest
[username@hostname ~]$ /usr/bin/time conda config --env --show root_prefix 2> /dev/null
root_prefix: /opt/miniconda/miniconda-latest
[username@hostname ~]$ conda create -n test conda
...
[username@hostname ~]$ conda activate test
(test) [username@hostname ~]$ conda config --env --show root_prefix 2> /dev/null
root_prefix: /opt/miniconda/miniconda-latest
(test) [username@hostname ~]$ /usr/bin/time conda config --env --show root_prefix 2> /dev/null
root_prefix: /home/drenshaw/.conda/envs/test
(test) [username@hostname ~]$ conda uninstall conda
...
(test) [username@hostname ~]$ conda config --env --show root_prefix 2> /dev/null
root_prefix: /opt/miniconda/miniconda-latest
(test) [username@hostname ~]$ /usr/bin/time conda config --env --show root_prefix 2> /dev/null
root_prefix: /opt/miniconda/miniconda-latestHere you can see the root_prefix changes depending on whether conda is installed in the active environment or not.
I get the above behaviour with conda 22.11.1 installed at the system level.
jaimergp
commented
1 year ago So the conda executable in $CONDA/envs/test/bin/conda is taking precedence before the conda shell function that allows env activation? 🤔 Or is it about the --env flag? That constitutes an issue too, but a different one than this I believe.
On Windows, conda (the executable) shadows conda (the shell function) in CMD. Not on PowerShell IIRC. That introduces nested environments and whatnot.
github-actions[bot]
commented
8 months ago Hi there, thank you for your contribution!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.
If you would like this issue to remain open please:
NOTE: If this issue was closed prematurely, please leave a comment.
Thanks!
jaimergp
commented
8 months ago not stale
Checklist
What happened?
I was using conda-libmamba-solver as my default solver. I installed in the base enviroment as recommended but recently I used the conda clean -a and I couldn't install anything more in the env that I was using. I guess it is because it cleaned all "unused" packages from that enviroment and conda-libmamba-solver was in the base env, which is different from the env I was using. It would probably be nice to change conda clean to keep the default solver (in this case Libmamba) from base intact instead of deleting it.
I took some time to figure it out what was wrong, so I imagine this is something users might have to deal with eventually when using conda clean command so I guess it is good to let you guys know besides being kind of obvious.
The enviroment info is from my "cf_python39" env, which is like my "default" enviroment, I install most of my daily packages available on conda-forge there. That's why so many packages installed.
Update: Maybe it was the config auto_activate_base false. I'm not sure anymore.
Conda info
Conda config
Conda list
Log file & additional context
There is no log report in this case, just this message.