search

alxmrs / xarray-sql

An experiment to query Xarray datasets with SQL
Apache License 2.0
25 stars 3 forks source link

Postgres/PostGIS integration #57

Open alxmrs opened 2 months ago

alxmrs commented 2 months ago

https://overcast.fm/+AAU1XJb7r0Y/9:25

Can Xarrat/Zarr be hooked up to PostGIS as out DB rasters?

Some notes:

alxmrs commented 2 months ago
alxmrs commented 2 months ago

Storage module (tiles) is bottleneck; lack of elastic parallelism.

https://overcast.fm/+AAU1XJb7r0Y/23:07

alxmrs commented 2 months ago

Further down this clip: row focus in query plan means a lack of efficient parallelism with raster data. To me, there is an analogy with Dask indexes, and the creation of special “pivoted raster” indexes as in https://github.com/alxmrs/xarray-sql/issues/26#issuecomment-1970822449.

alxmrs commented 1 month ago

It looks like TimeScale DB may be a good skew if Postgres to test an Xarray integration; they seem to have adopted a Zarr-like data model within Postgres internals.

https://github.com/timescale/docs.timescale.com-content/blob/master/introduction/architecture.md

cisaacstern commented 1 month ago

Second time timescale has come across my radar this week; haven't looked into it yet but intrigued!