locationtech / geowave

GeoWave provides geospatial and temporal indexing on top of Accumulo, HBase, BigTable, Cassandra, Kudu, Redis, RocksDB, and DynamoDB.
Apache License 2.0
502 stars 190 forks source link

For massive data, is there a sparse subsampling algorithm for line or polygon layers? #1561

Closed hsg77 closed 5 years ago

hsg77 commented 5 years ago

For massive data, is there a sparse subsampling algorithm for line or polygon layers? Your response to a discussion:: “Another rendering trick that really helps out with massive point data on a spatial index is SubsamplePoints but for this line/poly data that technique would produce some inconsistency.”

Point Layer I use the method of grid ellipsometry to get point elements (sub-sampling points) to get the top-level data of the pyramid looks good, but for line and polygon it is the opposite.

rfecher commented 5 years ago

I don't know what "grid ellipsometry" references. If you're using our "simple" subsampling technique, at least with a "tiered" index approach, the lines and polygons end up simply using their centroid as the skip point, resulting in potential visual inconsistencies if you were to rely on it for WMS rendering.

There is a way to do it without visual inconsistencies which I have protoyped, but its complex. Essentially we need to render the line/polygon data to a image mask in which each pixel of the mask represents a key. When a pixel is rendered in the mask, the key is removed from a sorted list and the top of the sorted list is the current valid skip row.

featuredistributed

hsg77 commented 5 years ago

Using "simple" subsampling technique to thinn the point-line polygon layer and save it as vector layer

(1)The grid thinning algorithm for point layers is adopted. (2)Douglas-Peucker thinning algorithm for line and polygon

hsg77 commented 5 years ago

In the 1.0 version of geowave, is it considered to build a vector pyramid for the layer with 100 million level layer data?

rfecher commented 5 years ago

for serving rendered maps from GeoServer, you can pre-render tiles using WMTS and geowebcache (seeding lower zoom levels while rendering higher zoom levels dynamically shouldn't be a problem)

By "vector pyramid," are you suggesting using vector tiles?

rfecher commented 5 years ago

Issue appears to be closed