ldeo-glaciology / xapres

package for processing ApRES data using xarray
MIT License
3 stars 2 forks source link

todo: use datatree to group data #9

Open jkingslake opened 1 year ago

jkingslake commented 1 year ago

cc @glugeorge

The idea it to use datatrees to group together data from ApRES instruments in the same survey.

@TomNicholas, this is the project we talked about that could be a good use case for datatrees. There are three zarrs stored in a google bucket, each containing similar data from different ice-penetrating radar units which ran for three months in 2022 (and are, hopefully, still running now!).

The zarrs have the same dimension names, but the coordinates have different values, particularly the time, AFGain, attenuator and filename coordinates.

Let us know if you think this is a good use case.

This code loads the data:

import xarray as np
def reload(site):
    filename = f'gs://ldeo-glaciology/apres/greenland/2022/single_zarrs/{site}'
    ds = xr.open_dataset(filename,
        engine='zarr', 
        chunks={}) 
    return ds
A101 = reload("A101')
A103 = reload("A103')
A104 = reload("A104')
TomNicholas commented 1 year ago

Sounds reasonable to me! Let me know if you have specific questions about datatree :slightly_smiling_face:

jkingslake commented 8 months ago

Still hoping to get round to this.

We also have multiple datasets from the same site that are potentially unaligned because the data was collected using different settings.

cc @glugeorge