Investigate Apache Arrow usage for the PDAL <-> JVM interaction

PDAL / java

Java extension and bindings for PDAL

https://pdal.io/java.html

Other

8 stars 10 forks source link

Investigate Apache Arrow usage for the PDAL <-> JVM interaction #21

Open pomadchin opened 5 years ago

pomadchin commented 5 years ago

Probably we can use Apache Arrow to simplify interaction between Java / C++ / Python / etc, however this requires some investigations.

[x] JMH Benchmarks that compare JVM Arrow to Java Collections
[ ] Prototype an example that works in Arrow (bindings API)
[ ] Benchmark the prototype

pomadchin commented 5 years ago

The next steps for this tasks would be:

to pack all the points taken from the pipeline into an apache arrow table
pass this into the JVM memory

NOTE: this experiment could be also a good test for the potential Python improvements as well, and mb it would be easier to exposer an Arrow table into Python rather than constructing a PyArray.