Ocean contour updates

[lbesnard]

@ocehugo added commits to your og PR https://github.com/aodn/imos-toolbox/pull/788 which can be closed.

The tests files used by the unittest need to be uploaded to the imos-toobox bucket, which I'll do once merged.

However I dug a bit in the sample files provided and had a bit of a concern.

The s500 sample file provided (s500_enu_avg.nc) seems to have a fillvalue of -32.7680 for the UCUR variable

The s1000 sample file (Avg_001.VTC.nc) on the other hand seems to have a fillvalue of -3.27680

If exported as a NetCDF, one can see that the valid_min and valid_max values for the UCUR variable are respectively -10 and 10.

So the repeated value of -3.27680 for the S1000 sample file ends up being in the valid range, but I believe it should be a FillValue.

Is there a reason why the s500 and s1000 seem to output a different fillvalue?

I don't have enough toolbox knowledge to know if this can be handled by the user in the metadata prior to the export

[evacougnon]

Looks good to me but before merging I'd prefer to wait for @hugo-sardi to comment on:

The s1000 sample file (Avg_001.VTC.nc) on the other hand seems to have a fillvalue of -3.27680 If exported as a NetCDF, one can see that the valid_min and valid_max values for the UCUR variable are respectively -10 and 10. So the repeated value of -3.27680 for the S1000 sample file ends up being in the valid range, but I believe it should be a FillValue. Is there a reason why the s500 and s1000 seem to output a different fillvalue?

[hugo-sardi]

In short, you don't need to worry about that, these values are dynamic, which is to be expected.

Is there a reason why the s500 and s1000 seem to output a different fillvalue?

They output a different value because they are different instruments/files. You can only deal with it since you can't change this output data.

I don't have enough toolbox knowledge to know if this can be handled by the user in the metadata prior to the export

Again, there is nothing extra to handle here. From the toolbox perspective, the data read from a file should be as is, and the only exception to that is to perform unit conversions. Everything else after that is handled by flags (for this case, see surface detection and friends) during the QC steps.

Further Explanation:

Firstly, these files do not apply any Fill to the data. The software explicitly creates netcdf variables with a _noFill attribute, and the actual _FillValue attribute doesn't exist. So, in the netcdf sense, there is no "FillValue/missng_value" in these files.

Secondly, in the netcdf land, a user with complete control of the variable creation step can define a custom fillValue. The user is free to select different fills for different variables/files. However, the oceanContour is not flexible since there is no such option to apply user defined NetCDF attributes.

Finally, my explanation here is that the data is different because there are innate characteristics between the instruments (e.g. frequency of operation). Although the processing applied is also different - one is not bin mapped, and the other is - the operating frequency is likely the origin since a different response magnitude at the sea surface is generated at these frequencies.