Add missing convention mandatory `Beam_groupX` variables to all instruments

[emiliom]

These 3 variables that are specified as mandatory in SONAR-netCDF4 v1 are currently missing from Beam_groupX, for all instruments (EK80, EK60, AZFP): beam_stabilisation, non_quantitative_processing, sample_time_offset. We'll add them.

@leewujung could you chime in with information about how we would populate the variables?

For beam_stabilisation, the EK80 manual describes the optional "Teledyne TSS1 datagram format" for "for heave, roll and pitch compensation", and the EK60 manual also refers to it ("Simrad Sounder/TSS1 protocol"). But it doesn't look like echopype handles it. Here's an extended discussion about this from the v1 convention manual: "The coordinate system for beam direction uses the same convention as the platform coordinate system, but with the origin at the transducer. Sonar beams can be compensated or uncompensated for platform motion. If uncompensated, the sonar beam co- ordinate system follows the yaw, pitch, and roll of the platform. If beams are compensated, the sonar beam coordinate system is translated and rotated per ping so that the y-axis is always horizontal and the z-axis is always vertical. The x-axis is always parallel to the platform's x-axis. The beam stabilisation variable indicates whether the beams are compensated or uncompensated."

non_quantitative_processing accommodates flags. Here's some description from the v1 convention: "Settings of any processing that is applied prior to recording backscatter data that may prevent the calculation of calibrated backscatter. A value of 0 always indicates no such processing.... [for the flag_values attribute] List of unique values (e.g. 0, 1, 3, 4) that indicate different non-quantitative processing settings that could be present in the sonar data. Must have the same number of values as settings given in the flag_meanings attribute." I'm guessing that we can just set it to 0 by default, unless you think there is relevant information present in the datagrams?

sample_time_offset: From the v1 convention: "Time offset applied to sample time-stamps and intended for applying a range correction (e.g. as caused by signal processing delays). Positive values reduce the calculated range to a sample."

[leewujung]

For beam_stabilisation: Yeah I don't think echopype handles that right now, so I think this would be not_stabilised = 0 (a side question: I remember trying to implement the enum type but got lost in it, are we just using True/False here or 1/0?)

For non_quantitative_processing: I agree with you that for echopype it'll just be 0

For sample_time_offset: the offset (or range_sample_offset) parameter from the raw data encodes this info for EK data, but it is not in the unit of time. If these are mandatory, should we just transform sample number to time?

[emiliom]

Re: beam_stabilisation and non_quantitative_processing: Thanks. I'll get to it.

a side question: I remember trying to implement the enum type but got lost in it, are we just using True/False here or 1/0?

Yeah, let's not try to bring some kind of enum type, which I don't think actually exists in the netcdf/CF/zarr world. I don't think there's a boolean type either, so I'll use np.byte.

For sample_time_offset: the offset (or range_sample_offset) parameter from the raw data encodes this info for EK data, but it is not in the unit of time. If these are mandatory, should we just transform sample number to time?

Hmm. I guess it does look that way. BTW, just to add to the definitions, here's the v1 long_name: "Time offset that is subtracted from the timestamp of each sample". The units are seconds. Is it straightforward to go from the sample_time_offset count to seconds??

[leewujung]

Is it straightforward to go from the sample_time_offset count to seconds??

Yes. It is the sample to time conversion, so essentially range_sample_offset * sample_interval = sample_time_offset, and we have sample_interval in Sonar/Beam_groupX for all instruments I believe.

[emiliom]

The conventions specifies a custom enum type for beam_stabilisation:

"byte enum beam_stabilisation_t {not_stabilised = 0, stabilised = 1}"

[emiliom]

range_sample_offset (offset) is available for EK's but not for AZFP. For AZFP we'll include it as a scalar with a value of 0.

[emiliom]

@leewujung can you confirm that we can close this issue now?

[leewujung]

Yes!