search

NCAS-CMS / cfa-conventions

NetCDF Climate and Forecast Aggregation (CFA) Conventions
https://github.com/NCAS-CMS/cfa-conventions/blob/main/source/cfa.md
1 stars 1 forks source link

Fragement variables and groups #40

Closed davidhassell closed 1 year ago

davidhassell commented 2 years ago

The conventions currently say of the address that "For each fragment, identifies the address of the fragment within its file.", however, they are not entirely clear on what to do if the fragment variable resides within a netCDF subgroup.

Example 7 implies that address elements should indeed contain group structures where needed (address = "/aggregation/temp1", "/aggregation/temp2" ;), but it is not clearly stated anywhere else.

I propose that we make it explicit that the netCDF group structure of an address must be included in its string, the one exception being if the address is in the root group then the leading / may be omitted.

Reasons for this are:

  1. If the group structure is omitted and the address variable name exists in multiple groups within the fragment file then it may be impossible to correctly build the aggregated data.
  2. It is excessively onerous to expect the application constructing the aggregated data to apply the CF group search algorithms to find the correct variable, notwithstanding reason 1.

Thanks, David

davidhassell commented 2 years ago

Hi @nmassey001 - any thoughts on this? Thanks.

nmassey001 commented 2 years ago

Hi David. Sorry for the late reply. Yes, I agree with making it explicit that the netCDF group must be supplied for the variable, unless it's the root group. This is (IIRC) how CFA-C works, so it might be worth seeing what happens in CFA-Python when specifying a variable in a group as an address.

davidhassell commented 2 years ago

Thanks, Neil, sounds good. Does that we need a new test?

nmassey001 commented 2 years ago

David, either a new test or a new example.

davidhassell commented 2 years ago

Thanks, Neil - with my previous comment I think that I might have thought we were over at CFA-Python!

I'm not so sure about my original proposal, now. I think that it is clear in the conventions on how to find aggregation variables, because the CF groups rules are assumed to apply just like everything else in CF. This should be stated in the text.

However, an implementation, like CFA-C will have to report on absolute group paths so that fragment variables can be unabiguously located. This means that CFA-C will need to implement the CF group search algorithms (if it doesn't already ...).

davidhassell commented 2 years ago

Suggested changes in #43

nmassey001 commented 2 years ago

David - can you point me towards the CF group search algorithms? I implement a search algorithm, but I want to make sure it matches up with the CF one. Thanks.

davidhassell commented 2 years ago

Sure. The text is at https://cfconventions.org/cf-conventions/cf-conventions.html#groups, which describes the three different search procedures that are in play: