Error upon trying calibration of Simrad EK80 .nc converted file: ValueError: dimensions ('channel',) must have the same length as the number of data dimensions, ndim=2

[icaro-00]

Hello all! I am currently trying to get sv values from Simrad EK80 echosounder. I converted the .raw file to .nc (netCDF4) files. I am now moving on the second step, where I am processing the .nc converted file and calibrate to get sv values. After updating to the 0.8.2 release, I do not get anymore the error in issue #1217, however, if I run the script provided, I run into another error:

ValueError: dimensions ('channel',) must have the same length as the number of data dimensions, ndim=2

I tried both files, that were shared in both #1217 and in #1220. They both return the same error mentioned above.

Does anyone know how to proceed to fix it? Or am I missing some mid-steps prior to calibration? Or?

The code I run:

nc_path = r"C:\Users\Runner\Desktop\Echopype\unpacked_files\20230925--D20230925-T082637.nc"

echodata = ep.open_converted(nc_path)

ds_Sv = ep.calibrate.compute_Sv(echodata, waveform_mode="CW", encode_mode="complex")

---

The error I get:

---

ValueError Traceback (most recent call last) Cell In[2], line 5 2 echodata = ep.open_converted(nc_path) # create an EchoData object 4 # For EK80 data, you need to specify waveform_mode and encode_mode ----> 5 ds_Sv = ep.calibrate.compute_Sv(echodata, waveform_mode="CW", encode_mode="complex")

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\api.py:206, in compute_Sv(echodata, kwargs) 120 def compute_Sv(echodata: EchoData, kwargs) -> xr.Dataset: 121 """ 122 Compute volume backscattering strength (Sv) from raw data. 123 (...) 204 it must be set using EchoData.update_platform(). 205 """ --> 206 return _compute_cal(cal_type="Sv", echodata=echodata, **kwargs)

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\api.py:60, in _compute_cal(cal_type, echodata, env_params, cal_params, ecs_file, waveform_mode, encode_mode) 58 # Perform calibration 59 if cal_type == "Sv": ---> 60 cal_ds = cal_obj.compute_Sv() 61 elif cal_type == "TS": 62 cal_ds = cal_obj.compute_TS()

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\calibrate_ek.py:628, in CalibrateEK80.compute_Sv(self) 619 def compute_Sv(self): 620 """Compute volume backscattering strength (Sv). 621 622 Returns (...) 626 and the corresponding range (echo_range) in units meter. 627 """ --> 628 return self._compute_cal(cal_type="Sv")

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\calibrate_ek.py:612, in CalibrateEK80._compute_cal(self, cal_type) 606 flag_complex = ( 607 True if self.waveform_mode == "BB" or self.encode_mode == "complex" else False 608 ) 610 if flag_complex: 611 # Complex samples can be BB or CW --> 612 ds_cal = self._cal_complex_samples(cal_type=cal_type) 613 else: 614 # Power samples only make sense for CW mode data 615 ds_cal = self._cal_power_samples(cal_type=cal_type)

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\calibrate_ek.py:546, in CalibrateEK80._cal_complex_samples(self, cal_type) 542 # Use pulse_duration in place of tau_effective for GPT channels 543 # below assumesthat all transmit parameters are identical 544 # and needs to be changed when allowing transmit parameters to vary by ping 545 ch_GPT = vend["transceiver_type"] == "GPT" --> 546 tau_effective[ch_GPT] = beam["transmit_duration_nominal"][ch_GPT].isel(ping_time=0) 548 # equivalent_beam_angle 549 # TODO: THIS ONE CARRIES THE BEAM DIMENSION AROUND 550 psifc = self.cal_params["equivalent_beam_angle"]

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\core\dataarray.py:879, in DataArray.setitem(self, key, value) 874 # DataArray key -> Variable key 875 key = { 876 k: v.variable if isinstance(v, DataArray) else v 877 for k, v in self._item_key_to_dict(key).items() 878 } --> 879 self.variable[key] = value

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\core\variable.py:881, in Variable.setitem(self, key, value) 879 value = Variable(dims[-value.ndim :], value) 880 # broadcast to become assignable --> 881 value = value.set_dims(dims).data 883 if new_order: 884 value = duck_array_ops.asarray(value)

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\core\variable.py:1481, in Variable.set_dims(self, dims, shape) 1478 else: 1479 expanded_data = self.data[(None,) (len(expanded_dims) - self.ndim)] -> 1481 expanded_var = Variable( 1482 expanded_dims, expanded_data, self._attrs, self._encoding, fastpath=True 1483 ) 1484 return expanded_var.transpose(dims)

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\core\variable.py:365, in Variable.init(self, dims, data, attrs, encoding, fastpath) 338 def init( 339 self, 340 dims, (...) 344 fastpath=False, 345 ): 346 """ 347 Parameters 348 ---------- (...) 363 unrecognized encoding items. 364 """ --> 365 super().init( 366 dims=dims, data=as_compatible_data(data, fastpath=fastpath), attrs=attrs 367 ) 369 self._encoding = None 370 if encoding is not None:

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\namedarray\core.py:252, in NamedArray.init(self, dims, data, attrs) 245 def init( 246 self, 247 dims: _DimsLike, 248 data: duckarray[Any, _DType_co], 249 attrs: _AttrsLike = None, 250 ): 251 self._data = data --> 252 self._dims = self._parse_dimensions(dims) 253 self._attrs = dict(attrs) if attrs else None

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\namedarray\core.py:480, in NamedArray._parse_dimensions(self, dims) 478 dims = (dims,) if isinstance(dims, str) else tuple(dims) 479 if len(dims) != self.ndim: --> 480 raise ValueError( 481 f"dimensions {dims} must have the same length as the " 482 f"number of data dimensions, ndim={self.ndim}" 483 ) 484 return dims

ValueError: dimensions ('channel',) must have the same length as the number of data dimensions, ndim=2

Thanks again for helping out!

[praneethratna]

@icaro-00 Thanks for reporting this, could you please confirm whether you are able to reproduce the error with raw file used in #1220? Because with the 0.8.2 release code, I'm not facing any error for that raw file on my local computer! Also, could you share the exact .raw file for which you are facing this error so that we can debug this better?

[icaro-00]

Hi @praneethratna thank you for your reply. I used the same .raw files I shared previously in the other two threads (#1220 and #1217). I converted them again to .nc but I still get errors when trying the calibration. Actually now the error is about: ValueError: shape mismatch: value array of shape (0, 1) could not be broadcast to indexing result with 1 dimensions

I noticed that this time, after converting the .raw file (20230925--D20230925-T082637) to .nc, the conversion still dropped the size of the file from 1GB, but this time instead of around 590MB, it converted to around 625MB.

Also, if I convert to .zarr and then calibrate the .zarr file, it works. So I am currently using .zarr format to move forward with my analysis.

The other error I got for the .nc file:

---

ValueError Traceback (most recent call last) Cell In[2], line 5 2 echodata = ep.open_converted(nc_path) # create an EchoData object 4 # For EK80 data, you need to specify waveform_mode and encode_mode ----> 5 ds_Sv = ep.calibrate.compute_Sv(echodata, waveform_mode="CW", encode_mode="complex")

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\api.py:206, in compute_Sv(echodata, kwargs) 120 def compute_Sv(echodata: EchoData, kwargs) -> xr.Dataset: 121 """ 122 Compute volume backscattering strength (Sv) from raw data. 123 (...) 204 it must be set using EchoData.update_platform(). 205 """ --> 206 return _compute_cal(cal_type="Sv", echodata=echodata, **kwargs)

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\api.py:60, in _compute_cal(cal_type, echodata, env_params, cal_params, ecs_file, waveform_mode, encode_mode) 58 # Perform calibration 59 if cal_type == "Sv": ---> 60 cal_ds = cal_obj.compute_Sv() 61 elif cal_type == "TS": 62 cal_ds = cal_obj.compute_TS()

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\calibrate_ek.py:628, in CalibrateEK80.compute_Sv(self) 619 def compute_Sv(self): 620 """Compute volume backscattering strength (Sv). 621 622 Returns (...) 626 and the corresponding range (echo_range) in units meter. 627 """ --> 628 return self._compute_cal(cal_type="Sv")

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\calibrate_ek.py:612, in CalibrateEK80._compute_cal(self, cal_type) 606 flag_complex = ( 607 True if self.waveform_mode == "BB" or self.encode_mode == "complex" else False 608 ) 610 if flag_complex: 611 # Complex samples can be BB or CW --> 612 ds_cal = self._cal_complex_samples(cal_type=cal_type) 613 else: 614 # Power samples only make sense for CW mode data 615 ds_cal = self._cal_power_samples(cal_type=cal_type)

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\echopype\calibrate\calibrate_ek.py:546, in CalibrateEK80._cal_complex_samples(self, cal_type) 542 # Use pulse_duration in place of tau_effective for GPT channels 543 # below assumesthat all transmit parameters are identical 544 # and needs to be changed when allowing transmit parameters to vary by ping 545 ch_GPT = vend["transceiver_type"] == "GPT" --> 546 tau_effective[ch_GPT] = beam["transmit_duration_nominal"][ch_GPT].isel(ping_time=0) 548 # equivalent_beam_angle 549 # TODO: THIS ONE CARRIES THE BEAM DIMENSION AROUND 550 psifc = self.cal_params["equivalent_beam_angle"]

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\core\dataarray.py:758, in DataArray.setitem(self, key, value) 753 # DataArray key -> Variable key 754 key = { 755 k: v.variable if isinstance(v, DataArray) else v 756 for k, v in self._item_key_to_dict(key).items() 757 } --> 758 self.variable[key] = value

File ~\anaconda3\envs\Echopype_v2.0\lib\site-packages\xarray\core\variable.py:839, in Variable.setitem(self, key, value) 837 value = as_compatible_data(value) 838 if value.ndim > len(dims): --> 839 raise ValueError( 840 f"shape mismatch: value array of shape {value.shape} could not be " 841 f"broadcast to indexing result with {len(dims)} dimensions" 842 ) 843 if value.ndim == 0: 844 value = Variable((), value)

ValueError: shape mismatch: value array of shape (0, 1) could not be broadcast to indexing result with 1 dimensions

[praneethratna]

Hey @icaro-00 I'm unable to reproduce the error with the test file i have. Could you please share the test files again so that we can look into this more closely? Thanks!

[icaro-00]

Hi @praneethratna, in the link here , you can find the link to the file.

Thanks for helping!

[ctuguinay]

@icaro-00 Sorry for the late response, but could you send over the file again? The link seems to have expired.

[leewujung]

@ctuguinay I have downloaded that file some time ago. I'll send it to you!

[leewujung]

@icaro-00 : it appears that the zip file I downloaded did not actually contain the test files but some weird unrelated images... Could you please upload again?

[leewujung]

Closing this as we can not access the file. @icaro-00 feel free to reopen.