Remove cfchecker from repo

[mauzey1]

We currently have cfchecker in the CMOR repo: https://github.com/PCMDI/cmor/tree/master/LibCV/cfchecker

It is not part of anything in CMOR or PrePARE, and has not been maintained. We should remove it from the repo.

[doutriaux1]

Let's move it to a separate repo.

[mauzey1]

Should we have a fork of https://github.com/cedadev/cf-checker in PCMDI instead?

[doutriaux1]

is that the same code?

[mauzey1]

It looks like the first version of cfchecks.py that got committed into CMOR's repo closely resembles cfchecks.py from cedadev from August 10, 2015.

The only differences added by our version is the change from self.AttrList['history']=['S','G'] to self.AttrList['history']=['S',('G', 'D')], and the addition of a couple print statements. Our version also recently got changes for Python 3 support but they haven't been tested since we don't use this code.

[doutriaux1]

thennthe other one is probably much better, let's just delete this one.

[hot007]

Speaking of this, we're trying to set up a publication env in Australia, and we're struggling to get PrePARE and cfchecks working in the same env - the cfchecks with CMOR was problematic, we replaced it with iris containing cfchecks but that broke PrePARE (different libnetcdf shared object files), we're working on resolving this, but per the Guidance for Modellers we are meant to be running both PrePARE and cfchecks so ideally do want them both in the same environment, does anyone have an env with both working in it, and if so what versions of each (and cfunits, udunits2) have you used?

[durack1]

@hot007 ouch this is a problem. @doutriaux1 do you know if the more recently nightly builds of CMOR would solve this problem (shared libraries)?

[hot007]

FWIW we can make both work at the same time by explicitly also loading a netcdf library with the correct shared object after activating the CMOR env, but ideally it should all work within the conda env without external dependencies like that.

[mauzey1]

@hot007 I tried installing both the nightly build of CMOR and cfchecker in conda.

conda create -n cmor_nighly_cfchecker -c pcmdi/label/nightly -c conda-forge cmor cfchecker python=2.7

When I tried to run cfchecks -h, I got this error:

Traceback (most recent call last):
  File "/home/mauzey1/anaconda3/envs/cmor_nighly_cfchecker/bin/cfchecks", line 7, in <module>
    from cfchecker.cfchecks import main
  File "/home/mauzey1/anaconda3/envs/cmor_nighly_cfchecker/lib/python2.7/site-packages/cfchecker/cfchecks.py", line 65, in <module>
    from cfunits import Units
  File "/home/mauzey1/anaconda3/envs/cmor_nighly_cfchecker/lib/python2.7/site-packages/cfunits/__init__.py", line 34, in <module>
    minimum_vn,  platform.python_version()))
ValueError: Bad python version: cf requires python version 3.0.0 or later. Got 2.7.15

When I tried to install CMOR and cfchecker with Python 3.7 it gave this error:

UnsatisfiableError: The following specifications were found to be in conflict:
  - cfchecker -> python[version='>=2.7,<2.8.0a0']
  - python=3.7

I get the ValueError: Bad python version: cf requires python version 3.0.0 or later. Got 2.7.15 error even if I install cfchecker in its own environment without CMOR.

I discovered that I needed to install cfchecker with cfunits 1.9 instead of 3.0.

conda create -n cmor_nightly_cfchecker -c pcmdi/label/nightly -c conda-forge cmor cfchecker python=2.7 cfunits=1.9

The Python version error for cfchecker went away, and cfchecks and the nightly version of PrePARE worked. I even tried it with CMOR 3.4.0 on conda-forge and got the same result. I didn't notice any issue being caused by the netcdf library. Could you show me how you set up the environment for PrePARE and cfchecker?

[hot007]

Thanks @mauzey1 , that's really helpful. I had simply used the CMOR 3.4 from conda-forge (which at this stage still comes with cfchecks), I didn't install cfchecker separately initially. But when cfchecks in the (released, not nightly) version of CMOR didn't work - we were getting cfunits errors - we added Iris (as we kind of wanted that too but it's not necessary to be in that env), which resolved the cfchecks problem but created the issue with netCDF libraries. I will try now with explicitly installing cfchecker and pinning the cfunits version, I suspect that'll be the trick, thank you!

[hot007]

Yes that worked, pinning cfunits to 1.9 I think was the key. Thank you @mauzey1 :)