conda-forge recipe

[bjlittle]

We should make the public ESMValTool easily available to the community by authoring a recipe on staged-recipes of conda-forge.

This will allow us to easily install the project as:

$ conda install -c conda-forge esmvaltool

[bjlittle]

Related to #5

[bulli92]

is there any convention for package naming in terms of UPPER/lowercase?

Thus should it be

conda install -c condo-forge ESMValTool

or

conda install -c condo-forge esmvaltool

??

[mkuzak]

I'm 100% in favour of all lower case.

[bulli92]

any need to start to compile a list of dependencies before generating the recipe? If so, where?

[bjlittle]

An example of the dependencies are here ... so that part is pretty much done. It's easy to create a conda recipe meta.yaml base on that to push to conda-forge staged-recipes for a new esmvaltool-feedstock ... but first we need to sort out a setup.py to make esmvaltool installable, ref #5.

Although, we do need to base the source for the recipe on either the REFACTORING/backend branch or tag a version of that branch. That's a decision that we need to agree on ... thoughts anyone? @ESMValGroup/esmvaltool-developmentteam

My name is a against #5, so really I should make time to do that i.e. get my finger out. That's the blocker here, the rest will follow ...

[bulli92]

o.k., in addition we would also need easytest. I am currently packing it for conda-forge.

[bjlittle]

@bulli92 What does easytest do that unittest, py.test, mock or testtools not already provide? I'm curious ...

[bulli92]

It's in principle a wrapper that uses unit test. Basic functionality is that it allows for comparison of directories, checking for file content, compare file content with reference data and so on ...

It's supposed to be used in a generic way for different projects.

For the ESMValTool testing I used it so far for to implement tests for different diagnostics that are based on synthetic data. The latter is used to allow for fast testing.

A preliminary implementation can be found here.

By the way, you had already a look on easytest after our workshop in Munich 2 years ago :-)

[bjlittle]

Ahh, I thought it sounded familiar! :wink: Does it have graphical testing folded in? I've now implemented graphical testing in iris based on perceptual image hashing, which makes it more resilient and robust, and less sensitive to changes in the software stack i.e. uping the version of matplotlib. This might come in handy for esmvaltool graphical testing ...

[bulli92]

Yes, I agree. Graphical testing is not implemented yet. The key challenge is that testing should be done efficiently for different diagnostics. Thus it currently allows to use either existing reference data or (current setup) generates tiny synthetic data snipets whic are then used by the diagnostics.

This is the starting point. We can do it more complex afterwards.

[bulli92]

dummydata would be needed as additional dependency. Will convert this in a condo-forge recipe soon.

[bouweandela]

Created a first draft here: https://github.com/ESMValGroup/ESMValTool/blob/REFACTORING_conda_recipe/meta.yaml

[fdiblen]

Created the Anaconda channel. https://anaconda.org/esmvalgroup

[valeriupredoi]

the conda feedstock will come in very handy for users to start complaining about environment and build issues, other than on the main gitHub issues page, I can try set up a test recipe in the next few days :beer:

[bouweandela]

Note that @zklaus has made some progress on this issue and esmvalcore as well as esmvaltool-python are now available on conda-forge.

[zklaus]

This is now available. The feedstocks are

• https://github.com/conda-forge/esmvalcore-feedstock
• https://github.com/conda-forge/esmvaltool-suite-feedstock

The packages are esmvalcore, esmvaltool-python, esmvaltool-r, esmvaltool-ncl, and esmvaltool.

[bouweandela]

Nice work @zklaus! You fixed by far our oldest outstanding problem :partying_face:

[valeriupredoi]

indeed, many x :beer: @zklaus