search

eurec4a / eurec4a-intake

Intake catalogue for EUREC4A field campaign datasets
17 stars 19 forks source link

Satellite products #37

Closed observingClouds closed 3 years ago

observingClouds commented 3 years ago

Hi, I thought I just add an entry about the satellite products that are available on AERIS. It's not all of them yet, since the naming of the variables is quite variable, but for now these might suffice to let me/and others create e.g. movies via opendap :)

leifdenby commented 3 years ago

I would go for cat.satellites.GOES16.regridded or maybe cat.satellites.GOES16.latlongrid? Or the description could mention what the data is regridded too. Mentioning the time resolution in the description could be helpful too.

Sounds good otherwise to me :)

tmieslinger commented 3 years ago

Great initiative! Thanks for getting the satellites started :) I agree with @d70-t and @leifdenby, cat.satellites.GOES16 seems straightforward and would make it easy to add further satellite products.

d70-t commented 3 years ago

I like @leifdenby's variant of cat.satellites.GOES16.latlongrid or similar, as obviously there could be other ways to regrid the dataset.

I am still wondering how to handle the product parameter. I currently see the following options:

The first (current) one seems to be more expressive which I like a lot. On the other hand, I see that the other file (i.e. for "2km_01min") are one file per minute. Thus I'd probably want to be able to express one of the following as well:

I am unsure if the former one could be implemented in the way, intake catalogs are currently working?

However, as I am going through the files, I am wondering if those are actually the same product and if channel is a sensible thing for that product? Probably this one is more like what it should look like?

Which then raises the question if product in this case is the right thing to call that variable (is it maybe something like resolution?) and also wether we actually need it? Probably it would be better to just get rid of it (at least for the moment) and if we later need it, we can insert a new parameter which defaults to the current behaviour. In that case, we'd probably end up with something like:

Which could potentially be extended in the future (if other resolutions became available) to support something like:

but the resolution parameter would stay optional and if it is left out, we'd default to what's necessary to support the current behaviour.

observingClouds commented 3 years ago

I like @d70-t last suggestion, because the products indeed differ. It would also be helpful, if one could define the paths according to the parameters. E.g. maybe not all of the products are on the same server or follow the same filename structure.

observingClouds commented 3 years ago

I updated the pull request. The regridded GOES16 satellite data can now be accessed by e.g.

from intake import open_catalog
cat = open_catalog("catalog.yml")
cat.satellites.GOES16.latlongrid(channel=2, date="2020-02-10").to_dask()
d70-t commented 3 years ago

Can you merge the current master into this branch? This should fix the failing check and we can proceed merging the PR.