eurec4a / eurec4a-intake

Intake catalogue for EUREC4A field campaign datasets
17 stars 19 forks source link

Platform tracks product #59

Closed observingClouds closed 3 years ago

observingClouds commented 3 years ago

This PR adds the track product of most (all?) platforms and closes #55 .

observingClouds commented 3 years ago

The SWIFT data that already existed (swift.yml) needs to be restructured as it does not allow for other products of those platforms to be added. @tmieslinger

observingClouds commented 3 years ago

Catalogs which use the opendap driver should mention that this catalog might require to import intake-xarray first

Done!

RobertPincus commented 3 years ago

Regarding the SWIFTs I can't quite parse what is bring proposed. How does one access the full data, and how the track data?

observingClouds commented 3 years ago

So currently, the swift data that consists of files like EUREC4A_ATOMIC_SWIFT16_All_v2.1.nc is accessed via

cat.swifts.SWIFT16

However, as discussed in the related issue #55 we like to access the platforms directly without any intermediate steps, e.g. it should always be

cat.<platform_id>

I would actually argue, that cat.<platform_id> should always be of type catalog, such that new data or products can easily be added to the catalog without changing its structure. So it should always be

cat.<platform_id>.<product>

In this specific case of the SWIFT buoys, it was not possible to add the tracks to the existing structure. The tracks are now available under cat.<platform_id>.track where <platform_id> is e.g. SWIFT16. So it would be great to put the "All" data entries into the respective yaml files, like SWIFT/SWIFT16.yaml and give the catalog entry a meaningful name.

To continue this discussion, we should probably create a new issue, since this is only partly related to this PR.

RobertPincus commented 3 years ago

I have no objection to revising the organization of the SWIFT data - I only want to be sure that this PR to add track information doesn't remove the SWIFT data already in the catalog.

observingClouds commented 3 years ago

No, don't worry @RobertPincus the swift data and the access structure are not affected by this PR.

leifdenby commented 3 years ago

Great to have these tracks in the catalog @observingClouds! Thanks!

My only reservation here @d70-t is that I think the root of the repository has too many folders now. Have you discussed moving all the platform yaml-file entrypoints into a sub-directory called say platforms/? This wouldn't change the catalog structure itself, just where the yaml-files are stored.

leifdenby commented 3 years ago

Or maybe something like ships/ gliders/ aircraft/ etc?

d70-t commented 3 years ago

😬 I was too quick...

We didn't discuss that. I've no specific preferences on that topic, but I have to admit that the github landing page looks a bit less obvious now :-)