search

NCAS-CMS / cf-python

A CF-compliant Earth Science data analysis library
http://ncas-cms.github.io/cf-python
MIT License
120 stars 19 forks source link

reading pp files with non-gregorian calendars does not work #170

Open bnlawrence opened 3 years ago

bnlawrence commented 3 years ago

I am attempting to aggregate as set of UM pp files in a directory. These include a file which appears to have a 30th of February. I had expected cf-python to warn me and change to a 360_day calendar (or allow me to specify it in the read statement).

Error message

 File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 678, in __init__
    c = self.time_coordinate(axiscode)
  File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2201, in time_coordinate
    vtimes = numpy_array([self.vtime(rec) for rec in recs], dtype=float)
  File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2201, in <listcomp>
    vtimes = numpy_array([self.vtime(rec) for rec in recs], dtype=float)
  File "/home/users/lawrence/miniconda3/envs/macbook/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2320, in vtime
    time = netCDF4_date2num(cftime.datetime(*LBVTIME),
 "cftime/_cftime.pyx", line 948, in cftime._cftime.datetime.__init__
  File "cftime/_cftime.pyx", line 1492, in cftime._cftime.assert_valid_date
ValueError: invalid day number provided in cftime.datetime(1992, 2, 30, 0, 0, 0, 0, calendar='gregorian')

Environment: sci6.jasmin.ac.uk, private conda install:

Platform: Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
HDF5 library: 1.10.6
netcdf library: 4.7.4
udunits2 library: /home/users/lawrence/miniconda3/envs/macbook/lib/libudunits2.so.0
python: 3.7.9
netCDF4: 1.5.3
cftime: 1.3.0
numpy: 1.19.4
psutil: 5.7.3
scipy: 1.5.3
matplotlib: 3.3.3
ESMF: not available
cfdm: 1.7.11
cfunits: 3.2.4
cfplot: 3.0.38
cf: 3.1.0
davidhassell commented 3 years ago

Hi Bryan, Could you let me know what the LBTIM header value is? The code is beholden to that - I'm most interested in the "units" part of the number (https://github.com/NCAS-CMS/cf-python/blob/master/cf/read_write/um/umread.py#L550-L558)

Thanks.

davidhassell commented 3 years ago

Hi Bryan - I think I may have worked this out - the default calendar behaviour of cftime changed at cftime 1.3.0. Assuming thaet this is the case, you either go back to cftime 1.2.0, or wait until we release cf-python 3.8.0 later this week. The new cf-python requires cftime>=1.3.0 and is adjusted accordinginly.

bnlawrence commented 3 years ago

You're too quick for me ... does this mean I don't need to find the LBTIM header? If so, I can wait for cf-python 3.8.0. Ta.

davidhassell commented 3 years ago

Yes - no need to go digging in the header, I think. It would be nice to test this theory out, though ... do you have a test file handy? Thanks.