search

paradedb / pg_analytics

DuckDB-powered analytics for Postgres
https://paradedb.com
PostgreSQL License
380 stars 15 forks source link

Geoparquet support #153

Open haubourg opened 1 month ago

haubourg commented 1 month ago

What feature are you requesting?

Hi, I'm testing pg_analytics for mostly geospatial use cases. Does pg_analytics supports geoparquet format as Crunchy bridge does ? I saw only GeoJSON examples in the docs and failed to do geographic subset filters when trying to mimic this example : https://www.crunchydata.com/blog/vehicle-routing-with-postgis-and-overture-data

Why are you requesting this feature?

For a massive datawarehouse postgres with mostly geospatial data, we go toward analytics and explore all the available options to outsource storage of data to S3 geoparquet, and benefit from column storage features for analytics.
After having screened most available extensions, pg_analytics seems really advanced and easy to set up.
GeoParquet is now an Open Geospatial Consortium standard (OGC) and along with GeoPackage should be the first class citizens when dealing with spatial data.

What is your proposed implementation for this feature?

GDAL and duckDB both support GeoParquet reading and could provide the necessary bits.

Full Name:

Régis Haubourg

Affiliation:

CSTB

philippemnoel commented 1 month ago

Hey @haubourg! Welcome :)

I don't believe we have Geoparquet support, but it should be easy to add by following our Parquet and GeoJSON supports if duckdb-rs supports it. Would you like to try your hand at it? We probably won't have time on our team to do this for quite some time.

haubourg commented 1 month ago

Thanks for your feedback! I unfortunately don't have the coding skills and my benevolent time goes to QGIS.org 😅. Regards

DeoLeung commented 1 week ago

duckdb's spatial extension >=1.1.0 supports limited geoparquet, at least writing and reading geometry type is ok

philippemnoel commented 1 week ago

duckdb's spatial extension >=1.1.0 supports limited geoparquet, at least writing and reading geometry type is ok

Thanks for letting us know! Would you be willing to submit a patch for it? It should be really small

DeoLeung commented 1 week ago

from python to rust requires some learning curve, just try cargo to bump it and ... learning why there are multiple versions in cargo.lock and ... the tests seems still installing v1.0.0 :(

philippemnoel commented 1 week ago

We are using duckdb-rs which is on duckdb 1.0.0. You' need to bump duckdb-rs to 1.1.0 before