Cant read file from Signature1000 with AHRS

[chrisawhitwell]

Hi,

Glad to see dolfyn has been rebuilt around xarray - thanks all for the hard work!

We had a recent field trip where we deployed 3 Nortek Signature1000's, however 2 of the 3 signatures wont open in dolfyn - see the traceback below. The instrument that made the files that wont open both have AHRS installed - is this not supported by the dolfyn i/o reader?

---

ValueError Traceback (most recent call last) Cell In[10], line 1 ----> 1 dat = dolfyn.read("......\Data\SWOT\W310\S100244A085_BB23_100244.ad2cp") 2 dat

File c:\Users\21310917\Anaconda\envs\research\Lib\site-packages\dolfyn\io\api.py:103, in read(fname, userdata, nens, kwargs) 99 func_map = dict(RDI=read_rdi, 100 nortek=read_nortek, 101 signature=read_signature) 102 func = func_map[file_type] --> 103 return func(fname, userdata=userdata, nens=nens, kwargs)

File c:\Users\21310917\Anaconda\envs\research\Lib\site-packages\dolfyn\io\nortek2.py:66, in read_signature(filename, userdata, nens, rebuild_index, debug, **kwargs) 62 raise TypeError('nens must be: None (), int, or len 2') 64 userdata = _find_userdata(filename, userdata) ---> 66 rdr = _Ad2cpReader(filename, rebuild_index=rebuild_index, debug=debug) 67 d = rdr.readfile(nens[0], nens[1]) 68 rdr.sci_data(d)

File c:\Users\21310917\Anaconda\envs\research\Lib\site-packages\dolfyn\io\nortek2.py:126, in _Ad2cpReader.init(self, fname, endian, bufsize, rebuild_index, debug) 124 self.f.seek(0, 2) # Seek to end 125 self._eof = self.f.tell() --> 126 self._index = lib.get_index(fname, 127 reload=rebuild_index, 128 debug=debug) 129 self._reopen(bufsize) 130 self.filehead_config = self._read_filehead_config_string()

File c:\Users\21310917\Anaconda\envs\research\Lib\site-packages\dolfyn\io\nortek2_lib.py:229, in get_index(infile, reload, debug) 227 out = np.fromfile(f, dtype=_index_dtype[index_ver]) 228 f.close() --> 229 _check_index(out, infile) 230 return out

File c:\Users\21310917\Anaconda\envs\research\Lib\site-packages\dolfyn\io\nortek2_lib.py:157, in _check_index(idx, infile, fix_hw_ens) 155 else: 156 hwe = idx['hw_ens'].copy() --> 157 period = hwe.max() 158 ens = idx['ens'] 159 N_id = len(uid)

File c:\Users\21310917\Anaconda\envs\research\Lib\site-packages\numpy\core_methods.py:41, in _amax(a, axis, out, keepdims, initial, where) 39 def _amax(a, axis=None, out=None, keepdims=False, 40 initial=_NoValue, where=True): ---> 41 return umr_maximum(a, axis, None, out, keepdims, initial, where)

ValueError: zero-size array to reduction operation maximum which has no identity

[jmcvey3]

Hi @chrisawhitwell, the inbuilt AHRS definitely is. What happens when you run dat = dolfyn.io.nortek2.read_signature("......\Data\SWOT\W310\S100244A085_BB23_100244.ad2cp", rebuild_index=True)? Same error?

[chrisawhitwell]

Hi @jmcvey3, yep I get the exact same error. Any other ideas of what might be the cause? See the deployment file below for some more context. AD2CP_1000kHz_900002_S245_W310_S245_internal_1ext_3s.zip

[jmcvey3]

Got it. Yes, a couple, stemming from that we've never tested an ADCP running an averaging interval in "narrowband" mode before.

I just pushed some code edits to #118 - could you pull that PR branch and run dolfyn.io.nortek2.read_signature("......\Data\SWOT\W310\S100244A085_BB23_100244.ad2cp", debug=True, rebuild_index=True) and let me know what the debugger log looks like and what error shows up next?

[chrisawhitwell]

I see, well I guess here is the opportunity! See the requested info below and let me know if there is anything else needed from my end.

Here is the dolfyn.log file: S100244A085_BB23_100244.dolfyn.log and the traceback now shows:

---

AttributeError Traceback (most recent call last) Cell In[2], line 1 ----> 1 dat = dolfyn.io.nortek2.read_signature("......\Data\SWOT\W310\S100244A085_BB23_100244.ad2cp",debug=True, rebuild_index=True) 2 dat

File c:\Users\21310917\Anaconda\envs\test_dolfyn_reader\Lib\site-packages\dolfyn\io\nortek2.py:66, in read_signature(filename, userdata, nens, rebuild_index, debug, **kwargs) 62 raise TypeError('nens must be: None (), int, or len 2') 64 userdata = _find_userdata(filename, userdata) ---> 66 rdr = _Ad2cpReader(filename, rebuild_index=rebuild_index, debug=debug) 67 d = rdr.readfile(nens[0], nens[1]) 68 rdr.sci_data(d)

File c:\Users\21310917\Anaconda\envs\test_dolfyn_reader\Lib\site-packages\dolfyn\io\nortek2.py:134, in _Ad2cpReader.init(self, fname, endian, bufsize, rebuild_index, debug) 131 self._ens_pos = self._index['pos'][lib._boolarray_firstensemble_ping( 132 self._index)] 133 self._lastblock_iswhole = self._calc_lastblock_iswhole() --> 134 self._config = lib._calc_config(self._index) 135 self._init_burst_readers() 136 self.unknown_ID_count = {}

File c:\Users\21310917\Anaconda\envs\test_dolfyn_reader\Lib\site-packages\dolfyn\io\nortek2_lib.py:487, in _calc_config(index) 485 # Now that we've confirmed they are the same: 486 config[id] = _headconfig_int2dict(_config[0], mode=type) --> 487 config[id].update(_beams_cy_int2dict(_beams_cy[0], id)) 488 config[id]['_config'] = _config[0] 489 config[id]['_beams_cy'] = _beams_cy[0]

AttributeError: 'NoneType' object has no attribute 'update'

[chrisawhitwell]

@jmcvey3 happy to share the binary if that makes it easier to troubleshoot. Let me know!

[jmcvey3]

@jmcvey3 happy to share the binary if that makes it easier to troubleshoot. Let me know!

@chrisawhitwell That would be great if you can do that! I think it's mostly a fileheader problem - the keywords for a couple parameters we get apparently only exist in the "burst" line, which this file doesn't have.

[jmcvey3]

@chrisawhitwell Thanks for sending that over - just pushed some fixes that'll read that file.

Some fixes weren't ideal - because Signature measurements from "averaging mode" are independent from "burst mode", most of the data variable names end in "_avg", including "vel_avg" and "time_avg". Instruments other than Nortek Signatures don't differentiate between burst and averaging mode, so this isn't an issue for those, but a lot of dolfyn functionality expects "vel" and "time", which this dataset didn't have. I copied "time_avg" to create "time", since that was needed off the bat, but if you'd like to use other functionality, you may need to drop the "_avg" tag from whichever variable names.

[chrisawhitwell]

Thanks @jmcvey3! I had another signature with the same sampling on this deployment that the fix worked for as well. Unsure if there is an elegant way around the naming system to be honest - maybe just a flag that can be passed to the reader initially by the user that nominates whether or not the averaging or burst mode variables should take the variable names expected by dolfyn?