Vendor-specific parameters

[ghost]

@leewujung commented on Jun 8, 2019, 1:41 PM UTC:

In implementing storing AZFP echosounder data into netCDF files, I run into the question of whether to store the various calibration parameters the manufacturer supplied with each of the transducer channels. Initially I wanted to store them into the Beam group, since adding attributes/variables are allowed under the convention. But there are so many of them so it seems a little messy. I am wondering if it is better to store these parameters in the Vendor-specific group. However this is not recommended in the convention since that group should only be used to store data that are not for quantitative analysis of the data.

Is there a recommendation on either of the solutions, or other recommendations, on this front?

PS. I did have to add a few parameters to the Beam group for EK60 data, but there are not that many of them.

This issue was moved by gavinmacaulay from ices-eg/wg_WGFAST#19.

[ghost]

@gavinmacaulay commented on Jun 9, 2019, 6:12 PM UTC:

I don't think it's a problem to have many additional variables in a group (before netCDF version 4 that was the only option). We could consider a sub-group under the Beam group.

Can you provide a list of what additional variables were needed to support AZFP and EK60 data? We can then get them into the next version of the convention. Having a documented naming scheme for AZFP calibration variables would be good, as would documenting the equation that converts from raw echosounder data to TS and Sv for AZFP data.

I very much want to keep the Vendor-specific group as the place where manufacturers (and format conversion software such as echopype) can dump whatever they like in whatever form they like and have it so that normal file reading code doesn't have to look in there for normal use.

[ghost]

@leewujung commented on Jun 9, 2019, 7:14 PM UTC:

Sounds good. I need a bit more time to finalize the code for AZFP. I'll put together a table with corresponding variables in AZFP's Matlab code and those written in the netCDF, along with explanation of each. EK60 is more straightforward but I'll make the same table. Will send the tables back here!

[ghost]

@cyrilleponcelet commented on Jun 11, 2019, 5:48 AM UTC:

Hi everyones, about the vendor-specific group, we faced the following problem : since dimension can only be inherited from a higher hierarchy you cannot retrieve any dimension like the ping_time in the /Vendor_specific group. To solve this problem we wanted to change convention and allow any group to have a sub group called _Vendorspecific with the same definition (dump what you want, the way you want, but not necessary for formal analysis). Furthermore this allow to have vendor_specific information organized with the same theme as the data (platform, environment, navigation).

What do you think of it ?

[ghost]

@gavinmacaulay commented on Jun 11, 2019, 8:45 AM UTC:

I think that is a good idea.

[ghost]

@leewujung commented on Jun 11, 2019, 7:40 PM UTC:

Ok great, I'll implement accordingly and link the correspondence table when done!

For AZFP some vendor-specific parameters are needed for echo integration but I think as long as they are in the Beam group it should be convenient and explicit enough, and can still be accessed under the same coordinate as cyrilleponcelet pointed out.