search

locationtech / rasterframes

Geospatial Raster support for Spark DataFrames
http://rasterframes.io
Apache License 2.0
246 stars 45 forks source link

`rf_agg_local_mean` error with ProjectedRaster type #333

Closed vpipkt closed 5 years ago

vpipkt commented 5 years ago

Initially saw this in doing a df.groupby.agg(rf_agg_local_mean('proj_raster)) in Python. The use of rf_tile was the immediate work around.

I don't understand the cast error, but I suspect a red herring here.

I will file a PR with a (ignored) unit test.

java.lang.ClassCastException: java.lang.Double cannot be cast to org.apache.spark.unsafe.types.UTF8String
    at org.apache.spark.sql.catalyst.expressions.BaseGenericInternalRow$class.getUTF8String(rows.scala:46)
    at org.apache.spark.sql.catalyst.expressions.GenericInternalRow.getUTF8String(rows.scala:195)
metasim commented 5 years ago

@vpipkt

I don't understand the cast error, but I suspect a red herring here.

FWIW, the error is technically on point because what's going on is that the struct layout of proj_raster has a double in the first field, but TileUDT expects the first field to be a string... basically a clear sign I missed this use case.