Add beam dimension to Beam group

[emiliom]

(Updated 2022-3-30). Add a beam dimension to Beam_groupX groups. We've decided to always include the beam dimension, instead of relying on an implicit length-1 dimension.

• [x] Currently for EK60 and AZFP, a beam dimension is not used explicitly and is effectively length 1. We'll need to add it.
• [x] #619 For EK80 there is already a dimension called quadrant that is conceptually equivalent (for our purposes) to the beam dimension as defined in SONAR-netCDF4 v1. We will need to rename it to beam.

Older notes (just for reference):

• For single beam or split-beam, it could be a length-1 explicit dimension or an implicit coordinate (scalar beam )
• For multi-beam, an explicit dimension is required. Currently the EK80 parser generates such a structure, but the dimension is named quadrant (WJ: there is an extra dimension quadrant — you can call them separate beams, but since Simrad use quadrant i used that for our first shot though I think it should be probably sector since it is not always 4)
• EM: It occurs to me now that, in terms of match-up with SONAR-netCDF4, the Beam group in AZFP and EK60 echopype nc/zarr files could be described as having an implicit, single-value beam dimension. Single-value dimensions can be implemented interchangeably in a netcdf file either explicitly with the dimension or implicitly (refer to the CF convention). In SONAR-netCDF4, since there is an explicit beam dimension, single-beam data would be expected to be stored with beam as a dimension.
• issue of convention use of beam dimension (ie, >1 beams per Beam subgroup? Based on "beam mode" alone?)
• In the convention, under 2.10.6 (Sonar groups), it says: "Variable definitions for data from split-aperture systems are not currently specified"

[leewujung]

For EK80:

transceiver_software_version

@leewujung wouldn't this variable be better placed in the Sonar group, where the convention attribute sonar_software_version is already found? I suggest moving it there, and not adding a ping_time dimension at this time; let's leave that addition to a future echopype version.

I agree with the above.

[b-reyes]

We have arrived at a consensus for EK80! Summary of dimension decisions for EK80.

EK80: Sonar/Beam_group1/Sonar/Beam_group2

• Variables having dims (channel, ping_time, range_sample, beam)
  • backscatter_i/r → backscatter_i is only in Sonar/Beam_group1
  • angle_alongship/athwartship → only in Sonar/Beam_group2
• Variables having dims (channel, ping_time, beam): these are vectors (dim: channel) in the data file and in reality does not vary across ping_time
  • beam_direction_x/y/z
  • angle_offset_alongship/athwartship
  • angle_sensitivity_alongship/athwartship
  • equivalent_beam_angle
  • beamwidth_twoway_alongship/athwartship
• To adhere to the convention, we will let beam_type have dims (channel, ping_time)
• Current echopype variables that do not need their dimensions modified
  • transducer_offset_x/y/z(channel) Note: this will be moved to the Platform group
  • transmit_duration_nominal(channel, ping_time)
  • transmit_power(channel, ping_time)
  • sample_interval(channel, ping_time)
  • slope(channel, ping_time)
  • transceiver_software_version(channel) Note: this will be moved to the Sonar group
• The variable channel_id will be “removed” in the future as it will become our coordinate channel see #566
• frequency_start/end(channel, ping_time) will have beam added as a dimension → only in Sonar/Beam_group1

[b-reyes]

Note that we will not be addressing the AD2CP sensor in this issue, this will be left for a future version of echopype. Thus, the last items to discuss are the AZFP sensor dimensions.

AZFP: Sonar/Beam_group1

• Variables having dims (channel, ping_time, range_sample, beam)
  • backscatter_r
• Variables having dims (channel, ping_time, beam): these are vectors (dim: channel) in the data file and in reality does not vary across ping_time
  • equivalent_beam_angle
  • gain_correction
• sample_interval should have dims (channel, ping_time), but currently echopype has it as just channel
• For transmit_duration_nominal we should add ping_time as a dimension to be consistent convention (it currently only has channel as a dimension)
• Variables not found in the convention, should we leave them alone?
  • temperature_counts(ping_time)
  • tilt_x/y_count(ping_time)
  • tilt_x/y(ping_time)
  • cos_tilt_mag(ping_time)
  • DS(channel)
  • EL(channel)
  • ​​TVR(channel)
  • VTX(channel)
  • Sv_offset(channel)
  • number_of_samples_digitized_per_pings(channel)
  • number_of_digitized_samples_averaged_per_pings(channel)

[leewujung]

The dimensions for AZFP listed above all look good. I just have some comments below on where specific form of data come from, and placement of specific variables.

• sample_interval should have dims (channel, ping_time), but currently echopype has it as just channel
• For transmit_duration_nominal we should add ping_time as a dimension to be consistent convention (it currently only has channel as a dimension)

In the same data file AZFP does not have the flexibility like EK systems to change sample_interval or transmit_duration_nominal over time (hence only a channel dimension straight from the data), but we could run into situations when combine_echodata is used and we would need to accommodate having different sample_interval over time, so having dims (channel, ping_time) is a convenient change for both variables.

• Variables not found in the convention, should we leave them alone?

I have some suggestions where the variables should be moved to. I was definitely not thinking about the variable placement earlier.

• Move from Beam_groupX to Vendor group:
  • the variables below are specific to AZFP as engineering outputs, some can be converted to "real" data variables in the Environment or Platform groups
  • temperature_counts(ping_time)
  • tilt_x/y_count(ping_time)
  • tilt_x/y(ping_time)
  • cos_tilt_mag(ping_time)
  • the variables below are specific to AZFP and used for AZFP-specific calibration
  • DS(channel)
  • EL(channel)
  • ​​TVR(channel)
  • VTX(channel)
  • Sv_offset(channel)
  • number_of_samples_digitized_per_pings(channel)
  • number_of_digitized_samples_averaged_per_pings(channel)

[b-reyes]

Based on our Thursday meeting, we have come to a consensus on the variable dimensions of the beam group for AZFP. To summarize, here are the final decisions on the dimensions for AZFP:

AZFP: Sonar/Beam_group1

• Variables having dims (channel, ping_time, range_sample, beam)
  • backscatter_r
• Variables having dims (channel, ping_time, beam): these are vectors (dim: channel) in the data file and in reality do not vary across ping_time
  • equivalent_beam_angle
  • gain_correction
• We will add ping_time to sample_interval and transmit_duration_nominal to comply with the convention. These variables will now have dims (channel, ping_time).
• The following variables are not found in the convention and their dimensions will not be modified. Note that these variables will be moved to other groups, see #642.
  • temperature_counts(ping_time)
  • tilt_x/y_count(ping_time)
  • tilt_x/y(ping_time)
  • cos_tilt_mag(ping_time)
  • DS(channel)
  • EL(channel)
  • ​​TVR(channel)
  • VTX(channel)
  • Sv_offset(channel)
  • number_of_samples_digitized_per_pings(channel)
  • number_of_digitized_samples_averaged_per_pings(channel)

[leewujung]

We are ready to close this now. 🎉