Event API - Githubissues

georust / geozero

Zero-Copy reading and writing of geospatial data.

Apache License 2.0

336 stars 33 forks source link

Event API #26

Closed pka closed 1 year ago

pka commented 2 years ago

This PR adds an event API, which should eventually replace the direct method call API. Main advandages are:

Support for chained processors (included are PromoteToMulti and LonLatToMercator)
GeomVisitor struct tracks main geometry type and collection flag, which can be used in format writers
Optional validation of event sequence
Easier serialization for sending events to other processes

Next steps:

Review API, escpecially exported modules
Compare performance with direct method call API (FlatGeoBuf writer)
Review FeatureProcessor API. Should it be also event based?

michaelkirk commented 2 years ago

I'm hoping this is related to https://github.com/georust/geozero/pull/22#issuecomment-1115135463

I'm eager to see how it shakes out - let me know if you'd like me to test something.

pka commented 2 years ago

It's slowly getting serious! Chaining like in #22 now supported.

pka commented 2 years ago

It's slowly getting serious! Chaining like in #22 now supported.

Nice! I've just taken a quick look at it, and it seems promising!

Do you think it's ready to attempt something like #36 (comment)? I can give it a try if you think so.

Will have to support the event API in a format writer first. What's your favorite output format you want apply reprojection? geo-types?

michaelkirk commented 2 years ago

Will have to support the event API in a format writer first. What's your favorite output format you want apply reprojection? geo-types?

geo-types would be great!

pka commented 2 years ago

Testing the event API by implementing it for the FlatGeoBuf writer resulted in some major changes (e.g. bd3f1dd) and the following open points:

processor type in GeomVisitor shouldn't be defined at compile time to support user flags like promote-to-multi yes/no
Performance of a combination of 3 processors (promote-to-multi, bbox and FGB feature writer) is currently 60% slower than the previous implementation (372us vs. 229us for countries.fgb)

Ideas to investigate:

New event type with coordinates slice instead of single coordinates
Compare with event API implementation as a single processor
Reduce size of event enum
Combine chained processors into single FSM

pka commented 1 year ago

Experimental branch is kept in https://github.com/pka/geozero/tree/event-api