Closed leewujung closed 1 year ago
Related to #1043.
We want to add: ['data_type', 'offset', 'count', 'channel_mode', 'pulse_form']
data_type
, count
and channel_mode
somewhere, such as the EK80 manual? That's to create a long_name
attribute. You've already described the other two variables.data_type
, offset
and count
seem overly generic.convert/parse_base.py
, it looks like all except data_type
have a channel
dimension only. For data_type
, it's not at all clear to me.Tagging @emiliom here for decisions on what to add and where. Note most of the already stored variables are in the
Sonar/Beam_group1
group even though those are not defined in the convention. We need to decide if to change the current implementation, or continue this approach to save those new variables in theSonar/Beam_group1
group also.
It seems like the primary rationale for adding these variables to the EchoData object is comprehensiveness, to retain all original, unique information in the converted dataset. Sticking all the variables in the Vendor_specific
group would address that goal. A secondary rationale is convenience: that is, if the variables are likely to be used somewhat frequently, having them in Beam_groupX
would help. In general, I'd advocate for only including a non-convention variable in Beam_groupX
if there are strong convenience reasons.
You could find information from here: Simrad EK80 reference manual These are under the Parameter XML datagram and RAW3/RAW4 datagram, and they can vary ping by ping.
It seems that data_type
(power, angle, complex) is something that we've already captured information for, so we don't have to store that. count
seems the same way. offset
on the other hand is the first sample number related to range calculation, so maybe we keep it, perhaps as range_sample_offset
or something similar.
Sample datagrams here: https://github.com/OSOceanAcoustics/echopype/blob/eb92185e83577b3e30bdcaf00e3e43ec0ada547a/echopype/convert/parse_base.py#L163-L229
It seems like the primary rationale for adding these variables to the EchoData object is comprehensiveness, to retain all original, unique information in the converted dataset. Sticking all the variables in the
Vendor_specific
group would address that goal. A secondary rationale is convenience: that is, if the variables are likely to be used somewhat frequently, having them inBeam_groupX
would help. In general, I'd advocate for only including a non-convention variable inBeam_groupX
if there are strong convenience reasons.
The parameters used for regular processing are pulse_form
, offset
, but if do not store data_type
and count
, then there's only one more channel_mode
, so I wonder if we should just keep all 3 in the Beam_groupX
group.
You could find information from here: Simrad EK80 reference manual
Thanks. As far as I can tell, that manual doesn't describe the datagram parameters. I found those descriptions, plus related useful information, in another manual: Simrad EK80 interface specifications
and they can vary ping by ping
Ah, ok.
The parameters used for regular processing are
pulse_form
,offset
, but if do not storedata_type
andcount
, then there's only one morechannel_mode
, so I wonder if we should just keep all 3 in theBeam_groupX
group.
It seems ok to store them in Beam_groupX
.
Here's what I've gleaned from the manual and your comments:
parameter name | new variable name | long_name | units | comments |
---|---|---|---|---|
pulse_form | pulse_form | Pulse type | - | An int code: 0 = CW, 1 = FM, 5 = FMD |
offset | range_sample_offset | First sample number | (a count) | |
channel_mode | channel_mode | - | An int code: 0 = Active, 1 = (not specified in manual; Inactive?) |
Oh I must have pasted the wrong link, but great that you were able to find the right one. Yeah it's the one that contains datagram format.
What you suggested looks great!
BTW, slope
doesn't currently have a long_name
. I looked through the manual and couldn't find a description. Any suggestions, while I'm at it?
I think it is in the manual under the Parameters XML diagram. It is the tapering applied to a square pulse. How about something like "A factor associated with the hann window used in generating the transmit signal " -- this is super specific, but that is where this parameter is used in.
Thanks. How about this more compact version: "Hann window slope parameter for transmit signal"
Hann window slope parameter for transmit signal"
Yes, this is 100% better than what I suggested!
PR #1094 fully addressed this issue (I hope!). We can close it now, right?
Yes, we can close this now -- but give me a chance to check a few more files for that offset
, and once I double confirm that we can close it.
I just realized that EK60 Beam_group1
has count
and offset
variables, too (and data_type
). Assuming they have the same meaning as in EK80, I suggest that for EK60 we rename offset
and drop count
(it's captured elsewhere?).
Oh I completely forgot about that. Good catch! Yeah I think renaming offset
and drop count
would be the way to go. Count is the number of bytes to read the data from I think, or something similar.
I think this issue has been fully addressed. @leewujung can you confirm that we can close the issue?
Yes, I think we can close this now!
Currently there are some ping data variables not stored in the converted EK files. These are not used at the moment but could be useful (see below). This issue is a reminder to add them after we have the calibration code is in good shape.
For EK80 each ping contains the following variables:
We want to add:
Note:
offset
is used for echo range calculation, though so far I've only seen files with offset=0 so currently there is no impact in not saving itpulse_form
: right now in the code we use the presence offrequency_start/end
to check which ping/channel is transmitting narrowband or broadband signals; probably more convenient to usepulse_form
, but that could cause backward compatibility issues.Already stored or captured in other forms:
Those we do not plan to store because they are already captured in other forms include:
Not sure:
Tagging @emiliom here for decisions on what to add and where. Note most of the already stored variables are in the
Sonar/Beam_group1
group even though those are not defined in the convention. We need to decide if to change the current implementation, or continue this approach to save those new variables in theSonar/Beam_group1
group also.