Can't read TDWR level 3 .nids files

[blaylockbk]

I'm getting TDWR files from the thredds server http://thredds.ucar.edu/thredds/catalog/terminal/level3/catalog.html

And can't seem to read in the file using the Level3File class. TDWR files must need to be decoded differently than NEXRAD files??

Get this error:

FILE = 'Level3_SLC_TV0_20150619_0831' f = Level3File(FILE) Traceback (most recent call last): File "", line 1, in File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/nexrad.py", line 1485, in init self.header = self._buffer.read_struct(self.header_fmt) File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/tools.py", line 152, in read_struct struct = struct_class.unpack_from(self._data, self._offset) File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/tools.py", line 45, in unpack_from return self._create(super(NamedStruct, self).unpack_from(buff, offset)) TypeError: unpack_from() argument 1 must be string or read-only buffer, not bytearray

[dopplershift]

Can you zip/gzip the data file in question and drag and drop it here? I can't reproduce the exact problem you're getting.

It works for me with TR0 from DEN. I can at least get some kind of failure with TV0 from SLC:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-2-436782dc74f3> in <module>()
----> 1 fobj = Level3File('/Users/rmay/Downloads/Level3_SLC_TV0_20160524_2321.nids')

/Users/rmay/repos/MetPy/metpy/io/nexrad.pyc in __init__(self, filename)
   1531         # Now that we have the header, we have everything needed to make tables
   1532         # Store as class that can be called
-> 1533         self.map_data = mapper(self)
   1534 
   1535         # Process compression if indicated. We need to fail

/Users/rmay/repos/MetPy/metpy/io/nexrad.pyc in __init__(self, prod)
    846             label = ''
    847             if codes >> 7:
--> 848                 label = self.lut_names[val]
    849                 if label in ('Blank', 'TH', 'ND'):
    850                     val = self.MISSING

IndexError: list index out of range

I'll dig into that, but I'd like to see if your problem is the same. Also, are you running off a git checkout of MetPy?

[blaylockbk]

I can't seem to get any .nids file to be read in. Here is one

Level3_SLC_TV0_20160516_2359.nids.gz

downloaded from: http://thredds.ucar.edu/thredds/catalog/terminal/level3/TV0/SLC/20160516/catalog.html?dataset=NWS/TDWR3/TV0/SLC/20160516/Level3_SLC_TV0_20160516_2359.nids

Executing this:

from metpy.io.nexrad import Level3File
f_TV0 = Level3File('Level3_SLC_TV0_20160516_2359.nids')

produces this error: Traceback (most recent call last): File "MetPy_read_TDWR.py", line 9, in f_TV0 = Level3File('Level3_SLC_TV0_20160516_2359.nids') File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/nexrad.py", line 1485, in init self.header = self._buffer.read_struct(self.header_fmt) File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/tools.py", line 152, in read_struct struct = struct_class.unpack_from(self._data, self._offset) File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/tools.py", line 45, in unpack_from return self._create(super(NamedStruct, self).unpack_from(buff, offset)) TypeError: unpack_from() argument 1 must be string or read-only buffer, not bytearray

We are using the metpy installed by pip, not a version from git

[dopplershift]

We are using the metpy installed by pip, not a version from git

I'm not sure I agree:

File "/uufs/chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/tools.py

Shows a version path of: 0.3.0+50.g3937f51--this reflects a git revision 3937f51 that I've not released yet; also, we only put source and wheels on pypi.python.org, not eggs.

That aside, if I roll back to python 2.7.3 (from 2.7.11), I can reproduce the error. I should be able to get this fixed. I was already planning a bugfix release, and I'll get this in.

[blaylockbk]

Oh, ok. I didn't do the install, so I wasn't sure which version we have. Thanks for looking into this

http://kbkb-wx.blogspot.com/ Brian

On Wed, May 25, 2016 at 12:13 PM, Ryan May notifications@github.com wrote:

We are using the metpy installed by pip, not a version from git

I'm not sure I agree:

File "/uufs/ chpc.utah.edu/sys/pkg/python/2.7.3_rhel6/lib/python2.7/site-packages/MetPy-0.3.0+50.g3937f51-py2.7.egg/metpy/io/tools.py

Shows a version path of: 0.3.0+50.g3937f51--this reflects a git revision 3937f51 that I've not released yet; also, we only put source and wheels on pypi.python.org, not eggs.

That aside, if I roll back to python 2.7.3 (from 2.7.11), I can reproduce the error. I should be able to get this fixed. I was already planning a bugfix release, and I'll get this in.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/metpy/MetPy/issues/148#issuecomment-221659322

[dopplershift]

Just to fill you in, I have a fix for this, I just need to get it committed (along with a few other things).

[blaylockbk]

Great news! Thanks for the quick fix

http://kbkb-wx.blogspot.com/ Brian

On Tue, Jun 14, 2016 at 9:15 AM, Ryan May notifications@github.com wrote:

Just to fill you in, I have a fix for this, I just need to get it committed (along with a few other things).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/metpy/MetPy/issues/148#issuecomment-225914787, or mute the thread https://github.com/notifications/unsubscribe/AF9cjXt5NljFXMES3XgInYOcDOMNvq25ks5qLsWDgaJpZM4IltoT .

[dopplershift]

TDWR fixes have landed--you may want to ask whoever did your install to re-install from git master. I'll also be cutting a MetPy release (0.3.1) some time today.