wslink throws error if variable info contains numpy float 32 values

[James-Crean]

wslink/websocket.py calls json.dumps() in order to transmit rpc results back to the JS client.

A user recently opened a .nc file where the bounds array contained numpy floats instead of the standard python version. json.dumps() cannot handle numpy 32bit floats and will throw an error, which prevents the file from being loaded.

[scottwittenburg]

I feel it should be the responsibility of our RPC methods to make sure they only return things which can be serialized as json. So do you know which RPC method (or methods) may be the culprit in this case?

[James-Crean]

Inside FileLoader.py

@exportRpc('cdat.file.variables')
def variables(self, file_name):

That method is currently the one having the issue. Not sure of other functions have a similar issue yet, but I imagine that this one might:

@exportRpc('cdat.file.var.info')
    def getvarinfofromfile(self, file_name, var_name=None):

[scottwittenburg]

By any chance, can you share the file causing the problem?

[James-Crean]

Sent you an email with the link to download it

[scottwittenburg]

@James-Crean I think #51 should start addressing this issue. But thanks to your suggestion I also tried the cdat.file.var.info RPC method on this data file and ran into another issue (seemingly with cdms2). Here is the python script to reproduce it:

import cdms2

tc_path = '<path-to-file-you-shared>'

f = cdms2.open(tc_path)

for variable in f.variables:
    print(variable)
    f(variable).info()

Here is the stack trace I got from that:

Traceback (most recent call last):
  File "../examples/varinfo.py", line 9, in <module>
    f(variable).info()
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/cdms2/cudsinterface.py", line 34, in __call__
    return v(*args, **kwargs)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/cdms2/avariable.py", line 170, in __call__
    grid=grid)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/cdms2/selectors.py", line 207, in unmodified_select
    grid=grid, raw=raw)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/cdms2/avariable.py", line 824, in subRegion
    return self.subSlice(*slicelist, **d)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/cdms2/avariable.py", line 646, in subSlice
    resultArray = self._returnArray(d, squeeze, singles=singles)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/cdms2/avariable.py", line 199, in _returnArray
    ar, mask=resultmask, fill_value=missing).astype(str)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/numpy/ma/core.py", line 2898, in __new__
    _data._fill_value = _check_fill_value(fill_value, _data.dtype)
  File "/data/scott/Documents/miniconda2/envs/cdat8/lib/python2.7/site-packages/numpy/ma/core.py", line 473, in _check_fill_value
    raise TypeError(err_msg % ndtype)
TypeError: Cannot set fill value of string with array of dtype |S1

[James-Crean]

Thanks for the script to reproduce the error!

I'll take a look at it, but most likely we will need to make an issue in cdms.