Conda package 2.0.0b4 broken?

[tommibergman]

I was trying to install the default version ESMValTool from Conda and it got jammed in infinitely collecting package metadata. Switching to 2.0.0b3 works better. Also notable is the size of 2.0.0b4 Conda package, which is only 23.4kB. Well I do not know how it is supposed to work, but default version, which I assume is 2.0.0b4, is not installing with Conda.

[valeriupredoi]

cheers for raising this @tommibergman - we are aware of this situation; a workaround can be found in the updated instructions here https://github.com/ESMValGroup/ESMValTool/pull/1675 - basically: install and link Julia, install esmvalcore and then install esmvaltool; this is happening because of conda's inability to solve the environment and is actually a problem with conda (it's trying to do much more than it's asked for); as for the size this is OK, @bouweandela has removed a lot of fluff from the conda build package as compared to previous versions :beer:

[bouweandela]

Indeed it's not broken, but conda is very slow to solve the environment (it took a bit more than an hour last time I tried https://github.com/ESMValGroup/ESMValCore/issues/660#issuecomment-637777906).

[bouweandela]

Pinning a single package as a hint to conda seems to really speed up the process, e.g. if I try to install with

conda create -n test -c esmvalgroup -c conda-forge esmvaltool python=3.8

it solves the environment in just 2 minutes, while

conda create -n test -c esmvalgroup -c conda-forge esmvaltool

takes over an hour.

[valeriupredoi]

yes @bouweandela found out the same workaround; I would advise installing esmvalcore then esmvaltool as per the instructions changefix in #1675 since that's a bit more straightforward for a user (you need esmvalcore package anyway). The reason why pinning and installing a large dependency in advance work the same way and alleviate conda's effort is that they are basically methods to reduce the degrees of freedom conda needs to account for when solving the environment :beer:

[bouweandela]

A new build of the package is now available on conda and installs in about 2 minutes on my computer. I had to label all older versions of the package with the deprecated label to make it work.

@tommibergman Could you check if this solves your problem?

[bouweandela]

Regarding the size difference between the v2.0.0b3 and v2.0.0b4 packages: in the b4 version we introduced subpackages, i.e. the esmvaltool is now just a placeholder with four dependencies esmvaltool-python, esmvaltool-r, esmvaltool-julia, esmvaltool-ncl, that's why it's so small.

[valeriupredoi]

legend, @bouweandela :beer: I shall test this myself now!

[tommibergman]

It is faster and looks good, but no cigar. esmvaltool-julia fails due to inability clone General.git:

LoadError: git repository not found athttps://github.com/JuliaRegistries/General.git`` I did a bit of digging, and it seems our proxy might be causing this.

Also R-packages fail to install: cannot open URL 'https://cloud.r-project.org/src/contrib/PACKAGES'

I suppose both errors occur due to our proxy, but I will investigate further next week.

[bouweandela]

Are you interested in running a particular recipe or all of them? If you don't need the R or Julia recipes, you can also install the tool with just

conda install esmvaltool-python esmvaltool-ncl -c conda-forge -c esmvalgroup

That will get about 70 percent of the recipes working.

[bouweandela]

@tommibergman Any news? Did you manage to solve the issue with the proxies? It would be interesting to know how you did it, because then we can add that as a hint to our installation instructions. Note that we released a new version of ESMValTool, v2.0.0 and a new conda package in the mean time.

[bouweandela]

Please re-open the issue if this is still a problem.