planetlabs / gpq

Utility for working with GeoParquet
https://planetlabs.github.io/gpq/
Apache License 2.0
144 stars 8 forks source link

Incrementally build converters and schema #12

Closed tschaub closed 1 year ago

tschaub commented 1 year ago

Previously, one feature at a time would be considered when building a Parquet schema from GeoJSON. This meant that feature collections with "sparse" properties (lots of nulls or missing properties in some features) could not reliably be converted to GeoParquet.

With this change, the convert command takes --min and --max arguments to control the minimum and maximum number of GeoJSON features to consider when building a schema. The schema is incrementally built using at least min features, and efforts to build a complete schema continue until max features.