Simplify earthio's readers - move to XArray data store back end class

ContinuumIO / earthio

Data reader utilities for machine learning on satellite imagery and Earth science data

32 stars 17 forks source link

Simplify earthio's readers - move to XArray data store back end class #12

Open PeterDSteinberg opened 7 years ago

PeterDSteinberg commented 7 years ago

Today we had an internal planning session among open source teams at Continuum and Ryan Abernathey of the Xarray team. One outcome was that I would reduce duplicated code between earthio and Xarray in file reading operations primarily by converting the functional style of earthio's data loaders to the Xarray back end data store classes' style.

PeterDSteinberg commented 7 years ago

Hi @gbrener - I assigned this one to you. We can discuss further at the meeting today.

PeterDSteinberg commented 7 years ago

Issue #17 is related to this issue, but deals with only the GeoTiff part of this issue. The fix for this issue should include making xarray backend data store classes that refactor:

earthio/tif.py
earthio/netcdf.py
earthio/hdf4.py
earthio/hdf5.py

Then we'll see what should be done about earthio.load_array and earth.load_meta (wrapper functions around the 4 modules listed above that tries to guess the right reader callable)

PeterDSteinberg commented 7 years ago

We discussed that each DataArray within the Dataset objects returned by these backends should be an dask.array for lazy access.