How to get access of each pings inside of a EchoData object ?

[Tchiimy]

Hello,

Sorry for the "noob" question, I'm new with this library and EK60 formats in general.

I'm trying to open a .raw file generated from my echosounder and get the number of pings and the data from the last one.

Using Pyecholab I succeed to do it:

from echolab2.instruments import EK60 ek60 = EK60.EK60()
  ek60.read_raw(file_path)
  print(ek60.n_pings)

  if ek60.n_pings > 0:
      # Accessing the last ping
      channel_id = ek60.channel_ids[0]
      raw_data = ek60.get_raw_data(channel_id=channel_id)
      last_ping =  raw_data.get_power()[-1, :]

But I don't succeed to do the same with echopype:

ed = ep.open_raw(
        file_path,
        sonar_model='EK60')
        print(ed)

I'm a little lost with the ed object, in theory where is stored the ping datas ?

I also got this warning:

/lib/python3.9/site-packages/echopype/utils/coding.py:87: 
UserWarning: Times can't be serialized faithfully to int64 with requested units 'seconds since 1900-01-01T00:00:00+00:00'. 
Resolution of 'microseconds' needed. 
Serializing times to floating point instead. 
Set encoding['dtype'] to integer dtype to serialize to int64. 
Set encoding['dtype'] to floating point dtype to silence this warning.
  encoded_data, _, _ = coding.times.encode_cf_datetime(

How am I suppose to correct this ?

[emiliom]

You can learn more about the EchoData object in the user manual, here. The ping data will be in the Sonar/Beam_group1 group, which you can access via ed["Sonar/Beam_group1"]

We can look into that time serialization warning. Can you share a small raw echosounder file that generates the warning?

Also, what version of echopype are you using? You can access the version via ep.__version__

[Tchiimy]

Thanks for the information !

Here is my sample: SCAN-D20231122-T172241.zip

echopype version:

import echopype as ep
print(ep.__version__)

0.8.2

Test with ed["Sonar/Beam_group1"]: import echopype as ep ed = ep.open_raw( file_path, sonar_model='EK60') print(ed["Sonar/Beam_group1"])

/home/pi/UPV_Proyecto_Prototypes/Proto_000_RPI_LED_RPI/UPV_LDP/lib/python3.9/site-packages/echopype/utils/coding.py:87: UserWarning: Times can't be serialized faithfully to int64 with requested units 'seconds since 1900-01-01T00:00:00+00:00'. Resolution of 'microseconds' needed. Serializing times to floating point instead. Set encoding['dtype'] to integer dtype to serialize to int64. Set encoding['dtype'] to floating point dtype to silence this warning.
  encoded_data, _, _ = coding.times.encode_cf_datetime(
/home/pi/UPV_Proyecto_Prototypes/Proto_000_RPI_LED_RPI/UPV_LDP/lib/python3.9/site-packages/echopype/utils/coding.py:87: UserWarning: Times can't be serialized faithfully to int64 with requested units 'seconds since 1900-01-01T00:00:00+00:00'. Resolution of 'microseconds' needed. Serializing times to floating point instead. Set encoding['dtype'] to integer dtype to serialize to int64. Set encoding['dtype'] to floating point dtype to silence this warning.
  encoded_data, _, _ = coding.times.encode_cf_datetime(
/home/pi/UPV_Proyecto_Prototypes/Proto_000_RPI_LED_RPI/UPV_LDP/lib/python3.9/site-packages/echopype/utils/coding.py:87: UserWarning: Times can't be serialized faithfully to int64 with requested units 'seconds since 1900-01-01T00:00:00+00:00'. Resolution of 'microseconds' needed. Serializing times to floating point instead. Set encoding['dtype'] to integer dtype to serialize to int64. Set encoding['dtype'] to floating point dtype to silence this warning.
  encoded_data, _, _ = coding.times.encode_cf_datetime(
/home/pi/UPV_Proyecto_Prototypes/Proto_000_RPI_LED_RPI/UPV_LDP/lib/python3.9/site-packages/echopype/utils/coding.py:87: UserWarning: Times can't be serialized faithfully to int64 with requested units 'seconds since 1900-01-01T00:00:00+00:00'. Resolution of 'microseconds' needed. Serializing times to floating point instead. Set encoding['dtype'] to integer dtype to serialize to int64. Set encoding['dtype'] to floating point dtype to silence this warning.
  encoded_data, _, _ = coding.times.encode_cf_datetime(
/home/pi/UPV_Proyecto_Prototypes/Proto_000_RPI_LED_RPI/UPV_LDP/lib/python3.9/site-packages/echopype/utils/coding.py:87: UserWarning: Times can't be serialized faithfully to int64 with requested units 'seconds since 1900-01-01T00:00:00+00:00'. Resolution of 'microseconds' needed. Serializing times to floating point instead. Set encoding['dtype'] to integer dtype to serialize to int64. Set encoding['dtype'] to floating point dtype to silence this warning.
  encoded_data, _, _ = coding.times.encode_cf_datetime(
<xarray.Dataset>

Dimensions:                        (channel: 1, ping_time: 110,
                                            range_sample: 657)
Coordinates:
  * channel                        (channel) <U24 '6601009 ZSR_120kHz B285M'
  * ping_time                      (ping_time) datetime64[ns] 2023-11-22T16:2...
  * range_sample                   (range_sample) int64 0 1 2 3 ... 654 655 656
Data variables: (12/27)
    frequency_nominal              (channel) float64 1.2e+05
    beam_type                      (channel) int64 0
    beamwidth_twoway_alongship     (channel) float64 11.0
    beamwidth_twoway_athwartship   (channel) float64 11.0
    beam_direction_x               (channel) float64 nan
    beam_direction_y               (channel) float64 nan
    ...                             ...
    transmit_duration_nominal      (channel, ping_time) float64 0.0003 ... 0....
    transmit_power                 (channel, ping_time) float64 350.0 ... 350.0
    data_type                      (channel, ping_time) int8 1 1 1 1 ... 1 1 1 1
    sample_time_offset             (channel, ping_time) float64 0.0 0.0 ... 0.0
    channel_mode                   (channel, ping_time) int8 0 0 0 0 ... 0 0 0 0
    backscatter_r                  (channel, ping_time, range_sample) float32 ...
Attributes:
    beam_mode:              vertical
    conversion_equation_t:  type_3

So if I understood the data is in ping_time ? Should I do something like print(ed["Sonar/Beam_group1"].sel(channel=ed["Sonar/Beam_group1"]["channel"][0],ping_time =ed["Sonar/Beam_group1"]["ping_time"][0]))

[leewujung]

@Tchiimy : I assume by "data" you mean the acoustic data? That is in ed["Sonar/Beam_group1"]["backscatter_r"]. "ping_time" is one of the dimensions of this data variable.

Many data variables reside in the ed["Sonar/Beam_group1"] xarray Dataset, and you can find more information on data variables, dimensions, and coordinates mean here.

The echopype-examples notebooks may be useful for some getting started stuff.

[emiliom]

@Tchiimy thanks for sharing the sample file. I've confirmed the time serialization warnings. We'll look into it.

[leewujung]

I've opened #1290 to track the serialization timestamp resolution issue, and we'll look into that this and next week. I'll close this issue now since that is not the core of this issue. @Tchiimy feel free to raise other issues if you run into other trouble.