[bug] 0.15.3 install package twice

[Legend94rz]

[wolfv]

can you give me the output of conda info?

[Legend94rz]

conda info:

     active environment : base
    active env location : /root/anaconda3
            shell level : 1
       user config file : /root/.condarc
 populated config files : /root/.condarc
          conda version : 4.10.3
    conda-build version : 3.21.4
         python version : 3.8.8.final.0
       virtual packages : __cuda=10.2=0
                          __linux=4.18.0=0
                          __glibc=2.27=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /root/anaconda3  (writable)
      conda av data dir : /root/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/noarch
          package cache : /root/anaconda3/pkgs
                          /root/.conda/pkgs
       envs directories : /root/anaconda3/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.8.8 Linux/4.18.0-2.4.3.kwai.x86_64 ubuntu/18.04.3 glibc/2.27
                UID:GID : 0:0
             netrc file : None
           offline mode : False

[LionTao]

Same here on windows when install pytorch with mamba 0.15.3

mamba install -n torch pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch

my conda info

(base) PS C:\Users\liontao> conda info

     active environment : base
    active env location : C:\Users\liontao\scoop\apps\miniconda3\current
            shell level : 1
       user config file : C:\Users\liontao\.condarc
 populated config files : C:\Users\liontao\.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.9.5.final.0
       virtual packages : __cuda=11.4=0
                          __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Users\liontao\scoop\apps\miniconda3\current  (writable)
      conda av data dir : C:\Users\liontao\scoop\apps\miniconda3\current\etc\conda
  conda av metadata url : None
           channel URLs : https://mirrors.sjtug.sjtu.edu.cn/anaconda/pkgs/r/win-64
                          https://mirrors.sjtug.sjtu.edu.cn/anaconda/pkgs/r/noarch
                          https://mirrors.sjtug.sjtu.edu.cn/anaconda/pkgs/main/win-64
                          https://mirrors.sjtug.sjtu.edu.cn/anaconda/pkgs/main/noarch
          package cache : C:\Users\liontao\scoop\apps\miniconda3\current\pkgs
                          C:\Users\liontao\.conda\pkgs
                          C:\Users\liontao\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\liontao\scoop\apps\miniconda3\current\envs
                          C:\Users\liontao\.conda\envs
                          C:\Users\liontao\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.9.5 Windows/10 Windows/10.0.19043
          administrator : False
             netrc file : None
           offline mode : False

[wolfv]

This will be solved once a new release with #1176 merged.

These mirrors return wrong data for the packages, currently.

[LionTao]

This will be solved once a new release with #1176 merged.

These mirrors return wrong data for the packages, currently.

Wow, that's great! Thank you!

[wolfv]

I hope this is resolved with 0.16.0! Please let me know if this issue persists

[SteveHawk]

I hope this is resolved with 0.16.0! Please let me know if this issue persists

The same issue still persists in 0.16.0, both on windows and linux. (I have tested windows 11 and wsl2 ubuntu 18.04)

One pattern I've noticed is that this issue only occurs when using a mirror (I have tried tsinghua and sjtu), and using a custom channel (I have tried conda-forge and pytorch) at the same time. Using defaults channel works fine.

Also, 0.14.* works fine, this behavior starts from 0.15.0.

Hope this can help diagnose the issue.

[wolfv]

Can you check the output when downloading some of the files from the curl command line?

I suspect the mirrors need to be updated and respect the new mamba user agent header

[wolfv]

Is 0.14 really still working fine?

[SteveHawk]

Can you check the output when downloading some of the files from the curl command line?

So I checked, downloading some conda packages from the mirror using curl works fine, and it appears that mamba is also downloading a fair amount of data from the mirror when installing, definitely not an error HTML page like #1169 shows.

Is 0.14 really still working fine?

Double checked, 0.14.0 and 0.14.1 both work fine.

But the fun one is 0.15.0, I realized that it's actually working, but the channel it's using is not the custom channel mirror specified in .condarc, but the original channel, as stated in #1080 .

After fixed in #1081 , all versions from 0.15.1 to 0.16.0 is using custom channel corretly, and is having this download twice issue. So I'm wondering if this fix is somehow causing this issue.

BTW, this is the .condarc I'm using:

default_channels:
  - https://mirror.sjtu.edu.cn/anaconda/pkgs/r
  - https://mirror.sjtu.edu.cn/anaconda/pkgs/main
custom_channels:
  conda-forge: https://mirror.sjtu.edu.cn/anaconda/cloud/
  pytorch: https://mirror.sjtu.edu.cn/anaconda/cloud/
channels:
  - conda-forge
  - defaults

[wolfv]

Thanks @SteveHawk it does sound a lot like our channel computation & custom channel implementation has some issue here.

[TTTPOB]

I think this issue still exists in my version:

>mamba --version
mamba 0.17.0
conda 4.10.3

>conda info
     active environment : None
            shell level : 0
       user config file : /home/username/.condarc
 populated config files : /home/username/.condarc
          conda version : 4.10.3
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __linux=5.10.60.1=0
                          __glibc=2.33=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/username/miniconda3  (writable)
      conda av data dir : /home/username/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/noarch
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/linux-64
                          https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2/noarch
          package cache : /home/username/miniconda3/pkgs
                          /home/username/.conda/pkgs
       envs directories : /home/username/miniconda3/envs
                          /home/username/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.8.5 Linux/5.10.60.1-microsoft-standard-WSL2 arch/ glibc/2.33
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

[wolfv]

hmm, how are you calling conda? You shouldn't mix these mirrors with the anaconda.org sources.

[TTTPOB]

hmm, how are you calling conda? You shouldn't mix these mirrors with the anaconda.org sources.

me? I aliased conda-forge to the mirror url and use mamba install -c conda-forge packagename I think there should be no mix since the only remote repo I use has been aliased to the mirror site

[wolfv]

@TTTPOB I do not see that in your configuration. Can you show your .condarc file?

[TTTPOB]

@TTTPOB I do not see that in your configuration. Can you show your .condarc file?

Oh, sorry, when I post this I think I might start to understand what you said by "mix these mirrors with the anaconda.org sources", does that mean I should not put anything in my default_channels?

anyway, my complete condarc has been pasted below

> cat ~/.condarc

auto_activate_base: false
channels:
  - defaults
show_channel_urls: true
default_channels:
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
custom_channels:
  conda-forge: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  msys2: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  bioconda: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  menpo: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  pytorch: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
  simpleitk: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud
# ssl_verify: false

[TTTPOB]

any progress on this? when creating massive conda environments, this is really frustrating bug

update: current version 0.19.1, this bug remains

[SteveHawk]

It looks like I might find a workaround.

So there's a configuration option available called channel_alias, according to conda's documentation:

Whenever you use the -c or --channel flag to give conda a channel name that is not a URL, conda prepends the channel_alias to the name that it was given. The default channel_alias is https://conda.anaconda.org.

If channel_alias is set to https://my.anaconda.repo:8080/conda/, then a user who runs the command conda install -c conda-forge some-package will install the package some-package from https://my.anaconda.repo:8080/conda/conda-forge.

The custom_channels option that we all have been using according to documentation is just an override:

custom_channels (map: primitive) A map of key-value pairs where the key is a channel name and the value is a channel location. Channels defined here override the default 'channel_alias' value.

The most popular anaconda mirrors available in mainland China right now (TUNA, SJTU, Aliyun) are all putting these custom channels under the same directory, namely https://<some domain here>/anaconda/cloud/. So I tried to set channel_alias instead of custom_channels, and the double installation issue just magically gone.

I have verified this method on my local machine (ubuntu 18.04) and in mambaforge docker, mamba version >= 0.18.0 all works fine without double installation. Here is a working example of .condarc:

channels:
  - defaults
show_channel_urls: true
default_channels:
  - https://mirrors.aliyun.com/anaconda/pkgs/main
  - https://mirrors.aliyun.com/anaconda/pkgs/r
  - https://mirrors.aliyun.com/anaconda/pkgs/msys2
channel_alias: https://mirrors.aliyun.com/anaconda/cloud/

Note that the down side of this workaround is that if a channel is not mirrored by the mirror site, mamba/conda will probably not be able to fall back to official anaconda channel.

[specter119]

Hi, is there any update on this?

@SteveHawk 's solution is excellent, however it can not resolve all issue. For the channels not maintained by the mirror site (like https://mirrors.aliyun.com/anaconda/), the channel_alias will prevent us from downloading the upstream packages (maintained in the https://anaconda.org/)

The recent version 0.22.1 the download twice issue still exists.

[TTTPOB]

any update on this?

[jonashaag]

Nobody is working on this so no use of regularly asking for progress. It is possible to use different URLs for each channel though, please read through the Conda docs.

[jonashaag]

Although we do not recommend to use the Anaconda channels at all, only conda-forge.

[specter119]

@jonashaag Somehing needs to be clarified:

It is possible to use different URLs for each channel though, ...

Yeah, setting channel_alias for the 3rd channels which mirror support, and reverting the rest to the upstream will work. However, this is just a temporary solution and doesn't solve this bug.

Although we do not recommend to use the Anaconda channels at all, only conda-forge.

Actually, users still need other channels like pytorch.

Currently (mamba 1.0.0), my setting doesn't include any mirror (custom_multichannels, custom_channels, and channel_alias), and the download twice bug still exists. Then looking forward to your early reply.