creare-com / podpac

Pipeline for Observational Data Processing Analysis and Collaboration
https://podpac.org
Apache License 2.0
45 stars 6 forks source link

Feature: get_source_data #485

Closed jmilloy closed 3 years ago

jmilloy commented 3 years ago

Datasource

>>> node = podpac.data.Array(
...             source=np.ones((3, 4)),
...             coordinates=podpac.Coordinates([range(3), range(4)], ["lat", "lon"]),
...         )
>>> 
>>> node.get_source_data()
<xarray.UnitsDataArray (lat: 3, lon: 4)>
array([[1., 1., 1., 1.],
       [1., 1., 1., 1.],
       [1., 1., 1., 1.]])
Coordinates:
  * lat      (lat) float64 0.0 1.0 2.0
  * lon      (lon) float64 0.0 1.0 2.0 3.0
Attributes:
    layer_style:   <podpac.core.style.Style object at 0x7f7c09fcb730>
    crs:           +proj=longlat +datum=WGS84 +no_defs +vunits=m
    geotransform:  (-0.5, 1.0, 0.0, -0.5, 0.0, 1.0)

TileCompositor

>>> a = ArrayRaw(source=np.arange(5) + 100, coordinates=podpac.Coordinates([[0, 1, 2, 3, 4]], dims=["lat"]))
>>> b = ArrayRaw(source=np.arange(5) + 200, coordinates=podpac.Coordinates([[5, 6, 7, 8, 9]], dims=["lat"]))
>>> c = ArrayRaw(source=np.arange(5) + 300, coordinates=podpac.Coordinates([[10, 11, 12, 13, 14]], dims=["lat"]))
>>> node = TileCompositorRaw(sources=[a, b, c])
>>> node.get_source_data()
<xarray.UnitsDataArray (lat: 15)>
array([100., 101., 102., 103., 104., 200., 201., 202., 203., 204., 300.,
       301., 302., 303., 304.])
Coordinates:
  * lat      (lat) float64 0.0 1.0 2.0 3.0 4.0 5.0 ... 10.0 11.0 12.0 13.0 14.0
Attributes:
    layer_style:  <podpac.core.style.Style object at 0x7f7bb627a340>
    crs:          +proj=longlat +datum=WGS84 +no_defs +vunits=m

Bounds

>>> node.get_source_data({"lon": (1.5, 4.5)})
<xarray.UnitsDataArray (lat: 3, lon: 2)>
array([[1., 1.],
       [1., 1.],
       [1., 1.]])
Coordinates:
  * lat      (lat) float64 0.0 1.0 2.0
  * lon      (lon) float64 2.0 3.0
Attributes:
    layer_style:   <podpac.core.style.Style object at 0x7f7bb6566190>
    crs:           +proj=longlat +datum=WGS84 +no_defs +vunits=m
    geotransform:  (1.5, 1.0, 0.0, -0.5, 0.0, 1.0)

closes #468

jmilloy commented 3 years ago

@mpu-creare This is a simple enhancement. It is just a general method implemented in the DataSource class and the TileCompositor class. Neither method calls eval, so it is a true enhancement and not just an API wrapper. However, some child datasource or datalib nodes could probably override get_source_data to be more efficient, such as perhaps the Rasterio node. At this point, I haven't done that (we can do that as needed as optimization), but I want to know if you would prefer to do some of that for this PR.