ANMN - global attributes missing

[ocehugo]

Under the devl project, I had to make some assumptions about the files and their contents, including presence of some global attributes. I got 267 files that "failed" to be detected as "IMOS ANMN" in my current test.

See the Log: anmn_ga_missing.txt

The testing global attributes is a limited set (based on DEVL requirements for metadata) but I think it can be useful for future reference.

Code for testing will be at in aodn/DEVL.

[mhidas]

Strange. Almost all of the files listed as missing "naming_authority" actually do have that global attribute, if you look on the AODN THREDDS server (e.g. http://thredds.aodn.org.au/thredds/fileServer/IMOS/ANMN/QLD/ITFTIS/Temperature/IMOS_ANMN-QLD_TZ_20120109T000000Z_ITFTIS_FV01_ITFTIS-1201-SBE56-127_END-20120723T000909Z_C-20180111T082021Z.nc)

[mhidas]

Actually, I can see those attributes fine on your THREDDS copy too. E.g. your list has

http://ci-eb-thredds-devl-data.aodn.org.au/thredds/dodsC/IMOS/ANMN/WA/WATR50/Velocity/IMOS_ANMN-WA_AETVZ_20151221T080000Z_WATR50_FV01_WATR50-1512-Long-Ranger-Workhorse-ADCP-496_END-20160720T031500Z_C-20180629T034621Z.nc is missing GA ['geospatial_vertical_min', 'geospatial_vertical_max']

But ...

$ ncdump -h http://ci-eb-thredds-devl-data.aodn.org.au/thredds/dodsC/IMOS/ANMN/WA/WATR50/Velocity/IMOS_ANMN-WA_AETVZ_20151221T080000Z_WATR50_FV01_WATR50-1512-Long-Ranger-Workhorse-ADCP-496_END-20160720T031500Z_C-20180629T034621Z.nc | grep geospatial_vertical_
        :geospatial_vertical_max = 505.3587f ;
        :geospatial_vertical_min = -2.467753f ;
        :geospatial_vertical_positive = "down" ;

[mhidas]

As far as I can see, the only files on your list actually missing those attributes are the 3 or 4 created back in the dark ages (2012), well before we started compliance checking, so not too surprising.

[ocehugo]

@mhidas , sorry attached the wrong log. I updated it with more keys missing.

naming_authority is missing in some of them. geospatial_vertical_min/max is missing is some.

[ocehugo]

The blamed time period appears to be 2007-2014 . Let me know if you want more details

[ocehugo]

ANMN files with missing "DEPTH" variables:

anmn_ga_no_depth_variable.txt

FYI, This is also for my self reference, given that i need to treat them with a different function.

[ggalibert]

ANMN files with missing "DEPTH" variables:

I know that some deployments didn't have a single valid pressure sensor on the mooring. In this case you can only rely on the nominal depth which is stored in global attribute instrument_nominal_depth and in variable NOMINAL_DEPTH.

[mhidas]

Thanks @ocehugo, that makes a bit more sense (for the missing global attributes). Again, almost all the files in that list have creation dates in 2012 (257 files), with only a few in 2013, and one in 2015. The solution is to re-process, which the facility has never been keen to do.

We could do a dodgy fix ourselves, but I would consider that an absolute last resort, only if this was proving to be a show-stopper for the DEVL project. Can you get by without these attributes, or these files?

[ocehugo]

I know that some deployments didn't have a single valid pressure sensor on the mooring. In this case you can only rely on the nominal depth which is stored in global attribute instrument_nominal_depth and in variable NOMINAL_DEPTH.

I know but in my limited mindset I think the DEPTH should be present in all mooring files, regardless of the type of instrument. I can't see a reason to avoid writing a DEPTH variable with the same size as TEMP or PSAL for example. Although one can think it's a waste of bits (say for "constant" depth - aqualoggers_T), the netcdf format is smart enough to not waste more than a few bytes for a constant vector.

@mhidas , not requesting fixes, just reporting :) .

I need to search/match/classify/store the moorings. At the moment, I'm just trying to filter/classifying stuff that I can/can't handle for DEVL (such as those with missing DEPTH, only with instrument_nominal_depth but not NOMINAL_DEPTH, etc).

As another layer of missing attributes on ANMN, here follows the files that are missing or "platform_code" or "deployment_code" - the only thing I can rely that exists in all files is "site_code".

anmn_spd_gatt.txt

[mhidas]

:+1: Thanks for reporting.

[ggalibert]

DEPTH has been used for measured depth. If it's not measured, it's not DEPTH.

platform_code and deployment_code are optional.