OSOceanAcoustics / echopype

Enabling interoperability and scalability in ocean sonar data analysis
https://echopype.readthedocs.io/
Apache License 2.0
99 stars 76 forks source link

[Data conversion] ValueError when attempting to import raw EK80 data #1413

Open tobitobitobi123 opened 1 week ago

tobitobitobi123 commented 1 week ago

General description of problem

Really excited to use echopype more once I have this running - thanks everyone. I have some EK80 data stored as .raw files (you can find a link to example files further down). When I attempt to open them using ep.open_raw I'm getting a ValueError. It seems it can't collect some information about the channel. I know that some of the channels/transducers ran in CW mode, some in FM mode, in case that's the issue. For another dataset that has a similar mixture of CW and FM modes there seems to be no issue.

Echopype version

echopype v0.9.0

How did you install Echopype (e.g., conda or pip)

conda

What is your operating system

Windows 10

Minimal code example

import echopype as ep
file = "DY150-D20220505-T060019.raw"
ed = ep.open_raw(file, sonar_model='EK80')

Error message printouts

Traceback (most recent call last):

  Cell In[5], line 1
    ed = ep.open_raw(file, sonar_model='EK80')

  File ~\anaconda3\envs\ek80\Lib\site-packages\echopype\utils\prov.py:237 in inner
    dataobj = func(*args, **kwargs)

  File ~\anaconda3\envs\ek80\Lib\site-packages\echopype\convert\api.py:536 in open_raw
    tree_dict["Vendor_specific"] = setgrouper.set_vendor()

  File ~\anaconda3\envs\ek80\Lib\site-packages\echopype\convert\set_groups_ek80.py:1367 in set_vendor
    ds = ds.pipe(self._add_filter_params, coeffs_and_decimation)

  File ~\anaconda3\envs\ek80\Lib\site-packages\xarray\core\common.py:775 in pipe
    return func(self, *args, **kwargs)

  File ~\anaconda3\envs\ek80\Lib\site-packages\echopype\convert\set_groups_ek80.py:1435 in _add_filter_params
    return dataset.assign(coeffs_xr_data)

  File ~\anaconda3\envs\ek80\Lib\site-packages\xarray\core\dataset.py:6160 in assign
    data.update(results)

  File ~\anaconda3\envs\ek80\Lib\site-packages\xarray\core\dataset.py:5026 in update
    merge_result = dataset_update_method(self, other)

  File ~\anaconda3\envs\ek80\Lib\site-packages\xarray\core\merge.py:1104 in dataset_update_method
    return merge_core(

  File ~\anaconda3\envs\ek80\Lib\site-packages\xarray\core\merge.py:761 in merge_core
    dims = calculate_dimensions(variables)

  File ~\anaconda3\envs\ek80\Lib\site-packages\xarray\core\variable.py:3282 in calculate_dimensions
    raise ValueError(

ValueError: conflicting sizes for dimension 'channel': length 4 on 'WBT_filter_i' and length 6 on {'channel': 'frequency_nominal', 'pulse_length_bin': 'sa_correction'}


### Example data

https://ln5.sync.com/dl/38297e4c0#pkq6pqwm-m6fiue6n-2hhax8d5-z3wugtpa

### Related existing issues or PRs

I have not seen a related issues - this just seems to be a relatively basic importing issue, possible having to do with my data.

### Troubleshooting steps

I've tried to replace some of the missing information with e.g. np.ones but this just caused more issues down the line.
leewujung commented 1 day ago

Hey @tobitobitobi123 : This is the same issue as #1317 -- it is a known issue and not a big one, but I haven't had a chance to get to it. I'll ping the person who would be working on it to see they can take on this soon. If not, would you be able to lend a hand on this?

tobitobitobi123 commented 23 hours ago

Hey @tobitobitobi123 : This is the same issue as #1317 -- it is a known issue and not a big one, but I haven't had a chance to get to it. I'll ping the person who said they would work on it to see they can take on this soon. If not, would you be able to lend a hand on this?

Oops - sorry, I overlooked #1317. I'd appreciate it if you could send a reminder to that person who wanted to work on it. I'm trying to convince my colleagues working with EK80 data that open source software like echopype can be the right choice for our purposes (instead of e.g. commercial software). However, because of this issue I couldn't access a whole dataset, which reduced my momentum significantly! And yes, I'd generally be able to lend a hand on this, but I can’t promise I’d be able to do so soon.