locationtech / rasterframes

Geospatial Raster support for Spark DataFrames
http://rasterframes.io
Apache License 2.0
243 stars 46 forks source link

TensorUDT prototype #434

Closed echeipesh closed 4 years ago

echeipesh commented 4 years ago

Prototype to use Arrow Tensor IPC message to add a tensor UDT.

The aim are twofold:

Its worth noting that in current world the bytes will still passed through python pickler on the way to python executors, which can't be great for performance.

echeipesh commented 4 years ago

Sorry for PR spam here, prototype PR against our fork, not worth attention ATM.