search

OSOceanAcoustics / echopype

Enabling interoperability and scalability in ocean sonar data analysis
https://echopype.readthedocs.io/
Apache License 2.0
96 stars 72 forks source link

Error upon trying calibration of Simrad EK80 .nc converted file: ValueError: zero-size array to reduction operation maximum which has no identity #1217

Closed icaro-00 closed 10 months ago

icaro-00 commented 11 months ago

Hello all! I am currently trying to get sv values from Simrad EK80 echosounder. I converted the .raw file to .nc (netCDF4) files. I am now moving on the second step, where I am processing the .nc converted file and calibrate to get sv values. However, if I run the script provided, I run into the error:

ValueError: zero-size array to reduction operation maximum which has no identity

Does anyone know how to proceed to fix it? Or am I missing some mid-steps prior to calibration? Or?

The code I run:

nc_path = r"C:\Users\Runner\Desktop\Echopype\unpacked_files\20230925--D20230925-T082637.nc"

echodata = ep.open_converted(nc_path)

ds_Sv = ep.calibrate.compute_Sv(echodata, waveform_mode="CW", encode_mode="complex")

And here below the error I get:

ValueError Traceback (most recent call last) Cell In[16], line 5 2 echodata = ep.open_converted(nc_path) # create an EchoData object 4 # For EK80 data, you need to specify waveform_mode and encode_mode ----> 5 ds_Sv = ep.calibrate.compute_Sv(echodata, waveform_mode="CW", encode_mode="complex")

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\api.py:206, in compute_Sv(echodata, kwargs) 120 def compute_Sv(echodata: EchoData, kwargs) -> xr.Dataset: 121 """ 122 Compute volume backscattering strength (Sv) from raw data. 123 (...) 204 it must be set using EchoData.update_platform(). 205 """ --> 206 return _compute_cal(cal_type="Sv", echodata=echodata, **kwargs)

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\api.py:60, in _compute_cal(cal_type, echodata, env_params, cal_params, ecs_file, waveform_mode, encode_mode) 58 # Perform calibration 59 if cal_type == "Sv": ---> 60 cal_ds = cal_obj.compute_Sv() 61 elif cal_type == "TS": 62 cal_ds = cal_obj.compute_TS()

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\calibrate_ek.py:628, in CalibrateEK80.compute_Sv(self) 619 def compute_Sv(self): 620 """Compute volume backscattering strength (Sv). 621 622 Returns (...) 626 and the corresponding range (echo_range) in units meter. 627 """ --> 628 return self._compute_cal(cal_type="Sv")

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\calibrate_ek.py:612, in CalibrateEK80._compute_cal(self, cal_type) 606 flag_complex = ( 607 True if self.waveform_mode == "BB" or self.encode_mode == "complex" else False 608 ) 610 if flag_complex: 611 # Complex samples can be BB or CW --> 612 ds_cal = self._cal_complex_samples(cal_type=cal_type) 613 else: 614 # Power samples only make sense for CW mode data 615 ds_cal = self._cal_power_samples(cal_type=cal_type)

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\calibrate_ek.py:501, in CalibrateEK80._cal_complex_samples(self, cal_type) 498 fs = self.cal_params["receiver_sampling_frequency"] 500 # Switch to use Andersen implementation for transmit chirp starting v0.6.4 --> 501 tx, tx_time = get_transmit_signal(beam, tx_coeff, self.waveform_mode, fs) 503 # Params to clarity in use below 504 z_er = self.cal_params["impedance_transceiver"]

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\ek80_complex.py:248, in get_transmit_signal(beam, coeff, waveform_mode, fs) 246 fs_chan = fs.sel(channel=ch).data if isinstance(fs, xr.DataArray) else fs 247 tx_params["fs"] = fs_chan --> 248 ych, = tapered_chirp(**tx_params) 250 # Filter and decimate chirp template 251 y_ch, y_tmp_time = filter_decimate_chirp(coeff_ch=coeff[ch], y_ch=y_ch, fs=fs_chan)

File ~\anaconda3\envs\Echopype\lib\site-packages\echopype\calibrate\ek80_complex.py:45, in tapered_chirp(fs, transmit_duration_nominal, slope, transmit_frequency_start, transmit_frequency_stop) 42 y[i0:i1] = y[i0:i1] w1 43 y[i2:i3] = y[i2:i3] w2 ---> 45 return y / np.max(y), t

File ~\anaconda3\envs\Echopype\lib\site-packages\numpy\core\fromnumeric.py:2810, in max(a, axis, out, keepdims, initial, where) 2692 @array_function_dispatch(_max_dispatcher) 2693 @set_module('numpy') 2694 def max(a, axis=None, out=None, keepdims=np._NoValue, initial=np._NoValue, 2695 where=np._NoValue): 2696 """ 2697 Return the maximum of an array or maximum along an axis. 2698 (...) 2808 5 2809 """ -> 2810 return _wrapreduction(a, np.maximum, 'max', axis, None, out, 2811 keepdims=keepdims, initial=initial, where=where)

File ~\anaconda3\envs\Echopype\lib\site-packages\numpy\core\fromnumeric.py:88, in _wrapreduction(obj, ufunc, method, axis, dtype, out, kwargs) 85 else: 86 return reduction(axis=axis, out=out, passkwargs) ---> 88 return ufunc.reduce(obj, axis, dtype, out, **passkwargs)

ValueError: zero-size array to reduction operation maximum which has no identity

Thanks a lot for helping out!

praneethratna commented 11 months ago

@icaro-00 Thanks for reporting this!, could you share the test file so that we can look into this more closely?

icaro-00 commented 11 months ago

Hi @praneethratna thanks for the reply!

Here you can download the converted file (.nc). The link expires in 7 days (FYI). :)

https://wetransfer.com/downloads/79771098f8c39a857e6e7e47baf7138a20231109164644/c81b55324b95fd1127dd3c78db042af320231109164658/193462?trk=TRN_TDL_01

I noticed that from the .raw to .nc, the file has shrunk of almost half, from more than 1GB to 590ish MB. Is that supposed to occur with the file conversion?

And I also noticed that for another converted .nc file, once I try to calibrate it, I get another issue: KeyError: transmit_type (but I can open another thread about it if I do not find any other issues previously debugged on Simrad EK80 of the same species).

Thanks a lot for the help!

praneethratna commented 11 months ago

I noticed that from the .raw to .nc, the file has shrunk of almost half, from more than 1GB to 590ish MB. Is that supposed to occur with the file conversion?

I'm not very sure of this but file size should supposedly decrease, @leewujung could you confirm this?

And I also noticed that for another converted .nc file, once I try to calibrate it, I get another issue: KeyError: transmit_type (but I can open another thread about it if I do not find any other issues previously debugged on Simrad EK80 of the same species).

Could you open an another issue for that error along with a link for the test file so that we can look into that? Thanks!

icaro-00 commented 11 months ago

I noticed that from the .raw to .nc, the file has shrunk of almost half, from more than 1GB to 590ish MB. Is that supposed to occur with the file conversion?

I'm not very sure of this but file size should supposedly decrease, @leewujung could you confirm this?

And I also noticed that for another converted .nc file, once I try to calibrate it, I get another issue: KeyError: transmit_type (but I can open another thread about it if I do not find any other issues previously debugged on Simrad EK80 of the same species).

Could you open an another issue for that error along with a link for the test file so that we can look into that? Thanks!

Hi @praneethratna I opened now another issue: https://github.com/OSOceanAcoustics/echopype/issues/1220

Thanks!

leewujung commented 10 months ago

I noticed that from the .raw to .nc, the file has shrunk of almost half, from more than 1GB to 590ish MB. Is that supposed to occur with the file conversion?

This is interesting! Usually we find the file size to expand rather than decreasing, but it may be that the data are very compressible given the particular default compression setting that we have in to_netcdf and to_zarr.