search

NCAS-CMS / cfa-conventions

NetCDF Climate and Forecast Aggregation (CFA) Conventions
https://github.com/NCAS-CMS/cfa-conventions/blob/main/source/cfa.md
1 stars 1 forks source link

Clarify the use of non-standardized aggregation instructions #41

Closed davidhassell closed 10 months ago

davidhassell commented 1 year ago

The CFA conventions at CFA-0.6.1 allows non-standardized aggregation instructions with the following text:

"The value of a term token identifying an aggregation instruction may be standardized or non-standardized, with the understanding that application programs should ignore terms that they do not recognise or which are irrelevant for their purposes. The purpose of allowing non-standardized tokens is to facilitate the aggregation of fragments stored in other file formats to those described by these conventions."

This is fine, but a discussion between @bnlawrence and @davidhassell (that was clearly also informed by others leading up to that!) highlighted another use case - to provide a means of storing metadata that relate to each fragment, but which are not necessary for the creation of the aggregated data.

In particular, it may be convenient for some metadata properties that are defined within the fragment files to be made available to the aggregated variable, without having to open and inspect the fragment files themselves - which may not be currently available to client reading the CFA-netCDF file.

Note that an array of metadata in this form does not comprise metadata as recognized by the CF data model, because it spans the fragment dimensions rather than those of the aggregated data, but an application could choose to implement it as a CF-compliant auxiliary coordinate variable by broadcasting the array values to the aggregated dimensions.

PR to follow to exemplify this use case.

davidhassell commented 1 year ago

Hi - @bnlawrence, @nmassey001, @sadielbartholomew, @JonathanGregory,

I'm revisiting the open CFA issues, and wondered if you had any comments on this suggestion and its PR (#42).

Many thanks, David

davidhassell commented 1 year ago

I'm currently thinking that it would be more natural to represent this metadata with a field ancillary construct, rather than with an auxiliary coordinate construct. This seems like a more natural home for metadata which are distributed over the same sampling domain as the field itself, but which are not domain-locating metadata. I shall update the PR accordingly.

davidhassell commented 1 year ago

... well, either might be desired, I suppose, so I'll say that in the text: https://github.com/NCAS-CMS/cfa-conventions/pull/42/commits/41ecb171f2bafea43ebec04af75799305f846206