What to do for data that is necessary, but not available?

[gavinmacaulay]

A concept behind the convention is that a conforming file contains all that is necessary to calculate absolute quantities, such as Sv or TS. Another concept was that the sonar/echosounder should be able to write conforming files, even if it lacks some variables, such as calibration parameters. These two concepts can conflict and the solution used in version 1.0 was to only apply the Mandatory variable requirement for things that the sonar/echosounder would always have and to have Mandatory if Available/Applicable on other variables that would be needed for absolute quantities.

However, there are situations where a sonar/echosounder doesn't have a Mandatory variable and the processing software can't readily provide a substitute.

This issue suggests a way to solve this awkwardness: introduce a Mandatory or Substitute requirement.

This means that if a variable that is mandatory, but no true or valid value is available, a meaningful (e.g., approximately correct) substitute can be put instead in the file.

This procedure already implicitly exists in other echosounder file formats, for example, the Sa correction in Simrad EK60 files is initially set to 0.0, which yields approximately correct absolute Sv values.

Comments?

[geoffmatt]

We have the same concept in Echoview. Pretty much all of the settings (calibration, environment etc) have a defined "default" value. If a calculation requires one of those settings, and they are not specified, then Echoview will automatically pull the defualt value. We will warn the user when that occurs so that they are aware that the value being used was not specified in the data. But that may not be possible here as I doubt there's a convenient place in the NetCDF format to specify that the included value is a "default"

[geoffmatt]

We already see this in our prototype code which exports to the NetCDF format. For example if someone is trying to export Simrad data but specifies the Furuno (type 2) conversion equations. In that case we need to use the Furuno parameters like Source Level to calculate the values which get written to the backscatter variables. However Source Level won't be specified, because the data is Simrad, and so Echoview will utilise the default value for that setting

[gavinmacaulay]

But that may not be possible here as I doubt there's a convenient place in the NetCDF format to specify that the included value is a "default"

We could add an attribute to the relevant variables to indicate whether they contain default/substitute values (e.g., defaulted=true). This has an advantage that it is a backwards compatible change - programs could ignore/use it as they see fit.

[geoffmatt]

That's true, and would definitely add information to the file contents which would be helpful when using the data

[gavinmacaulay]

ok, I'll create a PR to implement this.

[gavinmacaulay]

See #62 for a suggested way to address this issue.

[gavinmacaulay]

Done.