search

conda-forge / miniforge

A conda-forge distribution.
https://conda-forge.org/miniforge
Other
6.24k stars 323 forks source link

Default envs_dir for windows changed to %USERPROFILE%/.conda/envs somewhere after Miniforge 22.9 #529

Closed Krande closed 10 months ago

Krande commented 10 months ago

Solution to issue cannot be found in the documentation.

Issue

Hey,

On my Windows 10 machine the default location of the "envs" directory seems to have changed from <Miniforge3-installation-path>\envs to %USERPROFILE%/.condarc/envs somewhere between versions 22.9.0-1 to 22.11.1-4.

Due to security restrictions in the windows environment i'm currently on I am not allowed to install anything in the %USERPROFILE% directory. Basically anything installed there will not function properly, So when running the miniforge installer I install using "All Users" and choose a specific directory. I also allow creating start menu shortcuts and register Miniforge3 as the system python 3.10.

I was expecting that since I have explicitly chosen a "non-default" installation path, my created environments would be installed within the same installation directory. Or is that by design not the intention?

Here's what I have checked so far:

Tested across multiple versions/variants.

variant version result
Miniforge 23.3.1-0 %USERPROFILE%
Miniforge 23.3.1-1 %USERPROFILE%
Miniforge 23.1.0-4 %USERPROFILE%
Mini/Mambaforge 22.11.1-4 %USERPROFILE%
Mini/Mambaforge 22.9.0-1 <Miniforge3-installation-path>/envs

I use the miniforge prompt when I do mamba create -n testenv numpy to create a test environment.

Any help on this issue would much appreciated! Let me know if you need any more information to help on this,

Best Regards Kristoffer

Installed packages

# Name                    Version                   Build  Channel
boltons                   23.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310h00ffb61_0    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2023.7.22            h56e8100_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h628cb3f_3    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     23.3.1          py310h5588dad_0    conda-forge
conda-libmamba-solver     23.3.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
cryptography              41.0.3          py310h6e82f81_0    conda-forge
fmt                       9.1.0                h181d51b_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
jsonpatch                 1.32               pyhd8ed1ab_0    conda-forge
jsonpointer               2.0                        py_0    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
libarchive                3.6.2                h6f8411a_1    conda-forge
libcurl                   8.2.1                hd5e4a3a_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libiconv                  1.17                 h8ffe710_0    conda-forge
libmamba                  1.4.2                h8a7d157_0    conda-forge
libmambapy                1.4.2           py310h3fe4c2e_0    conda-forge
libsolv                   0.7.24               h12be248_3    conda-forge
libsqlite                 3.43.0               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libxml2                   2.11.5               hc3477c8_1    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              he774522_1000    conda-forge
mamba                     1.4.2           py310hd9d798f_0    conda-forge
menuinst                  1.4.19          py310h5588dad_1    conda-forge
miniforge_console_shortcut 1.0                  h57928b3_0    conda-forge
openssl                   3.1.2                hcfcfb64_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.4           py310h8d17308_1    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.10.12         h4de0772_0_cpython    conda-forge
python_abi                3.10                    3_cp310    conda-forge
reproc                    14.2.4               hcfcfb64_0    conda-forge
reproc-cpp                14.2.4               h63175ca_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.17.32         py310h8d17308_0    conda-forge
ruamel.yaml.clib          0.2.7           py310h8d17308_1    conda-forge
setuptools                68.1.2             pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   2.0.4              pyhd8ed1ab_0    conda-forge
vc                        14.3                h64f974e_17    conda-forge
vc14_runtime              14.36.32532         hfdfe4a8_17    conda-forge
vs2015_runtime            14.36.32532         h05e6639_17    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml-cpp                  0.7.0                h63175ca_2    conda-forge
zstandard                 0.19.0          py310h8d17308_0    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge

Environment info

active environment : base
    active env location : C:\Code\miniforge3
            shell level : 1
       user config file : %USERPROFILE%\.condarc
 populated config files : C:\Code\miniforge3\.condarc
                          %USERPROFILE%\.condarc
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=12.2=0
                          __win=0=0
       base environment : C:\Code\miniforge3  (read only)
      conda av data dir : C:\Code\miniforge3\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : C:\Code\miniforge3\pkgs
                          %USERPROFILE%\.conda\pkgs
                          %USERPROFILE%\AppData\Local\conda\conda\pkgs
       envs directories : %USERPROFILE%\.conda\envs
                          C:\Code\miniforge3\envs
                          %USERPROFILE%\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.3.1 requests/2.31.0 CPython/3.10.12 Windows/10 Windows/10.0.19044
          administrator : False
             netrc file : None
           offline mode : False
jaimergp commented 10 months ago

Can you post your conda config --show-sources?

Krande commented 10 months ago

Sure,

(base) C:\Users\XXX>conda config --show-sources
==> C:\Code\miniforge3\.condarc <==
channels:
  - conda-forge

==> C:\Users\XXX\.condarc <==
channels:
  - conda-forge
Krande commented 10 months ago

@jaimergp FYI: I tried on a 3rd windows 10 computer with mambaforge 23.1.0-4 already installed and for some reason the envs_dir order was as expected (ie. prioritizing <Miniforge3-installation-path>\envs)

 envs directories :
    C:\Code\miniforge3\envs
    %USERPROFILE%\.conda\envs
    %USERPROFILE%\AppData\Local\conda\conda\envs

Out of interest I wanted to see what happened if I uninstalled mambaforge 23.1.0-4 and installed Miniforge 23.3.1-1. And when checking conda info after installing miniforge, the envs_dir order is prioritizing %USERPROFILE% again.

 envs directories :
    %USERPROFILE%\.conda\envs
    C:\Code\miniforge3\envs
    %USERPROFILE%\AppData\Local\conda\conda\envs

How and where is the order of envs_dir set? I can try to look into this as long as I get a little nudge in the right direction?

Update:

I tried uninstalling Miniforge 23.3.1-1 and re-installing Mambaforge 23.1.0-4, but for some reason the order has not reverted back and is still prioritizing %USERPROFILE% :(

Update 2:

I tried running the miniforge prompt with admin privileges, and when I do the order is prioritizing <Miniforge3-installation-path>\envs. When I go back to "standard" privileges its back to %USERPROFILE%.

Update 3:

I think I've tracked it to a permission issue. It seems the installation directory is slapped with a "read-only" permission for Mini/Mambaforge > 22.9.X-X. So this (https://github.com/conda/conda/blob/main/conda/base/context.py#L650) fails.

Update 4:

It seems that Mini/Mambaforge 22.9.0-1 also slaps a "read-only" permission on the top level installation directory, and despite that still manages to get OS write permissions to append to the C:\Code\miniforge3\conda-meta\history.

Update 5 (hopefully solved):

Okay, it seems when I use the "Just me" option during the installation I DO get a choice of where I wish to install it. And when I use "Just me" I get the appropriate write permissions on C:\Code\miniforge3\conda-meta\history and all is well. I wonder if the option of choosing where I want to install for the "just me" option was not there in some of the older mambaforge installers? Or maybe its my poor memory.

Nonetheless, I believe this issue can be closed. I will try to install using the "just me" option on a few more windows PC's tomorrow to be certain that this isnt just a coincidence.

Krande commented 10 months ago

Nonetheless, I believe this issue can be closed. I will try to install using the "just me" option on a few more windows PC's tomorrow to be certain that this isnt just a coincidence.

I was able to successfully install with the expected envs_dir location using the "Just Me" on another windows 10 computer today, so you can close this issue if you want.

jaimergp commented 10 months ago

Thanks for the detailed investigation @Krande! When you install with "All Users", the installer ends up requesting admin permissions. But then if you run conda in a "Just me" context (e.g. regular cmd), then it won't be able to write to that base location. In a superuser console it would have, though. Glad you could find a solution!

Krande commented 10 months ago

@jaimergp Thanks! That makes a lot of sense.