geoschem / GEOSChem-python-tutorial

Python/xarray tutorial for GEOS-Chem users
Other
81 stars 34 forks source link

[FEATURE REQUEST] Let the regridder automatically skip those variables that do not contain latlon dims #7

Closed FeiYao-Edinburgh closed 4 years ago

FeiYao-Edinburgh commented 4 years ago

I have found that the new xesmf>=0.2.0 can regrid the whole dataset directly. However, for those dataset including variables that do not latlon dims, the program will complain. An example is the GEOS-Chem Restartfile that contains variables like hyam, hybm, hyai, hybi, P0. Therefore, I just wonder would it be great to let the xesmf detect those variables automatically and keep them the regridded file? Meanwhile, print some reminders/warnings that there are some variables do not support regridding?

Not clear about the internal achievement of regrid a dataset. If using a loop, then I suppose the following codes might be useful?

# supposing ds is the dataset that we want to regrid
result_list = [regridder_bilinear(dr) for varname, dr in ds.items() if 'lon' in dr.dims and 'lat' in dr.dims]
ds_result = xr.merge(result_list)

Moreover, incorporating ds_result['hyam'] = ds['hybm'] and the like?

yantosca commented 4 years ago

Thanks for writing. Right now the xESMF package only regrids DataArray variables and not Dataset variables. In the meantime, it is probably a good idea only to loop over the variables that you want to regrid, as you have done.

I will defer to @JiaweiZhuang, who is the maintainer of the xESMF package for his opinion. He might have a solution in mind.

JiaweiZhuang commented 4 years ago

However, for those dataset including variables that do not latlon dims, the program will complain. An example is the GEOS-Chem Restartfile that contains variables like hyam, hybm, hyai, hybi, P0

The easiest way is probably dropping those violating variables using ds.drop.

xESMF "loops" over a dataset via xarray.apply_ufunc that automatically handles chunking, parallelization, metadata tracking, etc. It's not a normal for loop where you can insert those additional checkings.

FeiYao-Edinburgh commented 4 years ago

Right now the xESMF package only regrids DataArray variables and not Dataset variables.

It seems that the latest xESMF is capable of regridding a Dataset variable.

In the meantime, it is probably a good idea only to loop over the variables that you want to regrid

I also think so. Just wonder if this can be achieved by providing an interface (e.g. uses provide a list of variables that they want to regrid) since I believe xarray.apply_ufunc is much quicker than users' loop? @JiaweiZhuang What do you think?

The easiest way is probably dropping those violating variables using ds.drop.

Cool!

yantosca commented 4 years ago

OK, I wasn't aware the latest xESMF can regrid a dataset.

You can either use ds.drop, or the "drop_variables" keyword to xr.open_dataset (or xr.open_mfdataset). For example:

ds = xr.open_dataset(myfile, drop_variables=['hyai', 'hybi', 'hyam', 'hybm', 'P0'])