Reading-eScience-Centre / pycovjson

Create CovJSON files from common scientific data formats
BSD 3-Clause "New" or "Revised" License
12 stars 7 forks source link

ISSUE-45 Implement exception handling for time object conversions #46

Closed lewismc closed 6 years ago

lewismc commented 6 years ago

This issue addresses #45. The new behavior is as follows

lmcgibbn@LMC-056430 /usr/local/pycovjson(ISSUE-42) $ pycovjson-convert -i SSS_OI_7D_20151482015154_V40.nc -o SSS_OI_7D_20151482015154_V40.nc_sss.covjson -v sss
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
Failed to decode times encoded in the standard NetCDF datetime format into datetime objects. Attempting less strict opening logic. Reason:  unable to decode time units 'Julian days since December 31, 2010' with the default calendar. Try opening your dataset with decode_times=False.
Successfully opened dataset with less strict logic.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error parsing and converting 'time' variable object to CovJSON compliant string.  Unknown string format
Constructing Range from variable: sss
Variable 'sss' has no axis attribute, executing fallback for manual axis value detection.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Manually detected axis as: '['t', 'y', 'x']'
Variable 'sss' has no axis attribute, executing fallback for manual axis value detection.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Error: DataArray does not include 'standard name' or 'name'.
Manually detected axis as: '['t', 'y', 'x']'
Attempting to write CovJSON manifestation to 'SSS_OI_7D_20151482015154_V40.nc_sss.covjson'
Completed in: '0.11765500000000007' seconds.

So although this product is still messy, we do provide the flexibility to successfully execute a conversion.

RileyWilliams commented 6 years ago

Hi @lewismc I will review this soon, could you send me one of the NetCDF files which causes this issue so that I can add a test for it? Thanks

lewismc commented 6 years ago

could you send me one of the NetCDF files which causes this issue so that I can add a test for it?

@RileyWilliams sure here. You can download the netCDF4 version

lewismc commented 6 years ago

I should note, that these files are not always CF compliant.. I have a feeling that is the reason ValueError's can be encountered. In reality however not all netCDF data is CF compliant so IMHO the pycovjson toolkit should provide flexibility as a priority.