ices-eg / wg_WGFAST

Working Group on Fisheries Acoustics, Science and Technology
11 stars 16 forks source link

Compliance checkers to verify AcMeta and SONAR-netCDF4 conventions #56

Closed Haris-Kunnath closed 4 years ago

Haris-Kunnath commented 4 years ago

It would be useful if we could provide compliance checker(s) possibly written in Python, R, or Matlab to check compliance of NetCDF (as an example) with latest AcMeta and SONAR-netCDF4 convention documents.

From an implementation point of view, a user at the end of the day may want to check whether the data generated is compliant with these conventions. This would help foster data sharing and reuse in future.

Asking from AcMeta point of view, is there any way to automatically harvest all global attributes listed in the document and potentially store in a CSV or some other format? Or any trick to quickly read these attributes (and associated obligation fields) from the HTML or XML file.

We have an in-house Matlab function (with preconditions) written for checking AcMeta convention along with other metadata authorities – not elegant but did warn the user about missing global and variable attributes as you may see in the attached image.

Cheers

-Haris netcdf_checker

gavinmacaulay commented 4 years ago

I agree that a conformance checker would be useful and quite likely necessary. For ease of use, a online version might be best, but this could just be a front-end to something in python, R, etc.

There is cfchecker.py, which checks for CF compliance, which sonar-netcdf4 tries to conform to. But this isn't checking anything specific to acmeta or sonar-netcdf4.

To automatically harvest the acmeta attributes, I suggest turning the problem around - store the attribute tables as csv files(s) and then include them into the asciidoc file (asciidoctor will accept tables as delimiter-separated values). There might be some gotcha's to do with formatting of the tables for asciidoctor (see here), but hopefully not crucial.

ghost commented 4 years ago

This issue was moved by gavinmacaulay to ices-publications/SONAR-netCDF4#19.