Currently the UDTs work fine in PySpark, and UDFs if called from SQL context. Need to figure out the Python constructs and configuration necessary to register UDFs and UDAFs with the Python DataFrame API.
Support will need to be done in two phases. The first addressing the UDF/UDAF registration/declaration in the DataFrame context. The second is to figure out how to declare and enforce the RasterFrame constraints around spatial key, tile layer metadata, and tiles. This is likely much harder without a type system, perhaps requiring custom Python wrappers.
metasim
commented
7 years ago
Currently the UDTs work fine in PySpark, and UDFs if called from SQL context. Need to figure out the Python constructs and configuration necessary to register UDFs and UDAFs with the Python DataFrame API.
Support will need to be done in two phases. The first addressing the UDF/UDAF registration/declaration in the DataFrame context. The second is to figure out how to declare and enforce the RasterFrame constraints around spatial key, tile layer metadata, and tiles. This is likely much harder without a type system, perhaps requiring custom Python wrappers.