search

SlideRuleEarth / sliderule-python

SlideRule Earth Example Notebooks: On-demand, cloud-based processing of satellite mission data (NASA ICESat-2, GEDI, ArcticDEM/REMA, HLS)
https://slideruleearth.io/rtd/
BSD 3-Clause "New" or "Revised" License
41 stars 21 forks source link

Standardized response considerations #72

Closed jpswinski closed 1 year ago

jpswinski commented 2 years ago

From Scott:

This is not something currently in SlideRule, but we could consider ways in which the results of SlideRule can be made Cloud-friendly and utilize some emerging standards: Specifically I'm thinking about how sliderule results are saved. We typically want to work with a GeoDataFrame, but what about saving that to share 'downstream' with other software or colleagues. The example notebooks suggest saving to HDF5. What if there were an option to stage on S3 for some limited period of time as Zarr or Parquet with accompanying STAC metadata https://github.com/stac-extensions/pointcloud?

jameshgrn commented 2 years ago

Adding a suggestion for feather files instead of parquet for short term fast io. Also natively read by geopandas

jpswinski commented 1 year ago

SlideRule v2.1.0 now supports returning results as GeoParquet files which can be streamed back to the user or written to an S3 bucket for later retrieval. See the outputs parameters in the SlideRule documentation for more details.