search

doneill / koop-provider-pg

A postgis provider for koop
Apache License 2.0
13 stars 6 forks source link

failed process was running: #16

Closed jjhi11 closed 2 months ago

jjhi11 commented 4 months ago

When testing locally and connecting to my cloud postgresql instance I see this error in the logs.

Failed process was running: SELECT jsonb_build_object( 'type', 'FeatureCollection', 'features', jsonb_agg(features.feature) ) FROM ( SELECT jsonb_build_object( 'type', 'Feature', 'gid', 'gid', 'geometry', ST_AsGeoJSON(ST_Transform(shape,3857))::jsonb, 'properties', to_jsonb(inputs) - 'shape' ) AS feature FROM (SELECT * FROM public.seamlessunitstestmarshall) inputs) features;

The url I was trying is http://localhost:8080/pg/rest/services/public.seamlessunitstestmarshall/FeatureServer

jjhi11 commented 4 months ago

I assume this is the issue? {"error":{"code":500,"message":"error: total size of jsonb array elements exceeds the maximum of 268435455 bytes on public.seamlessunitstestmarshall","details":[]}}

My data is too large.

doneill commented 2 months ago

Right, using JSONB which has a limit of 255MB. Will consider a chunking approach for large datasets 🤔

doneill commented 2 months ago

Added a PG_LIMIT environment variable to allow retrieving a portion of large datasets