search

Reading-eScience-Centre / pycovjson

Create CovJSON files from common scientific data formats
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

Various issues when using pycovjson-convert #30

Closed lewismc closed 7 years ago

lewismc commented 7 years ago

Hi Folks, I'm pasting some CMD activity for review... This concerns use of pycovjson-convert with THIS GRANULE. I use the nc4 manifestation. I'm going to dig deeper on all of this in due course as I think the tool needs to be more robust. Not all data conforms to CF conventions... this is just how things are. I think pycovjson would be a much more useful tool if it offers flexibility.

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v sea_surface_temperature -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-sea_surface_temperature.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: sea_surface_temperature
Warning: Variable sea_surface_temperature has no axis attribute.
Error in axes_dict
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 52, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 65, in _construct_coverage
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 125, in _construct_range
TypeError: 'NoneType' object is not iterable

...

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v analysed_sst -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-analysed_sst.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: analysed_sst
Warning: Variable analysed_sst has no axis attribute.
Warning: Variable analysed_sst has no axis attribute.
Converting....
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 56, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 187, in _save_covjson
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 208, in save_json
TypeError: dumps() takes 1 positional argument but 2 positional arguments (and 2 keyword-only arguments) were given

...

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v analysis_error -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-analysis_error.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: analysis_error
Warning: Variable analysis_error has no axis attribute.
Warning: Variable analysis_error has no axis attribute.
Converting....
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 56, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 187, in _save_covjson
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 208, in save_json
TypeError: dumps() takes 1 positional argument but 2 positional arguments (and 2 keyword-only arguments) were given

...

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v latitude -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-latitude.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: latitude
Warning: Variable latitude has no axis attribute.
Error in axes_dict
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 52, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 65, in _construct_coverage
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 125, in _construct_range
TypeError: 'NoneType' object is not iterable

...

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v longitude -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-longitude.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: longitude
Warning: Variable longitude has no axis attribute.
Error in axes_dict
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 52, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 65, in _construct_coverage
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 125, in _construct_range
TypeError: 'NoneType' object is not iterable

...

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v time -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-time.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: time
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 52, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 65, in _construct_coverage
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 151, in _construct_range
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/read_netcdf.py", line 164, in get_values
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

...

lmcgibbn@LMC-056430 /usr/local/playground/public/coverages(master) $ pycovjson-convert -i nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4 -v mask -o nc_20160304-DMI-L4UHfnd-NSEABALTIC-v01-fv01-DMI_OI.nc.gz.nc4-mask.covjson
Error in sitecustomize; set PYTHONVERBOSE for traceback:
KeyError: 'PYTHONPATH'
wrote variable: mask
Warning: Variable mask has no axis attribute.
Warning: Variable mask has no axis attribute.
Traceback (most recent call last):
  File "/Users/lmcgibbn/miniconda3/bin/pycovjson-convert", line 11, in <module>
    load_entry_point('pycovjson==0.3.9', 'console_scripts', 'pycovjson-convert')()
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/cli/convert.py", line 65, in main
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 52, in write
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 66, in _construct_coverage
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/pycovjson-0.3.9-py3.6.egg/pycovjson/write.py", line 101, in _construct_params
  File "/Users/lmcgibbn/miniconda3/lib/python3.6/site-packages/xarray/core/common.py", line 168, in __getattr__
    (type(self).__name__, name))
AttributeError: 'DataArray' object has no attribute 'units'
RileyWilliams commented 7 years ago

@lewismc I agree, more flexibility would definitely be good, I haven't worked on the project for a while but am planning to get back to it soon.

lewismc commented 7 years ago

ack @RileyWilliams I'm going to go ahead and hammer some of these issues out.