locationtech / rasterframes

Geospatial Raster support for Spark DataFrames
http://rasterframes.io
Apache License 2.0
240 stars 46 forks source link

Missing rf_local_biased_add in pyrasterframes #578

Open lepitrust opened 2 years ago

lepitrust commented 2 years ago

I found this function on scala source code, but not working on pyrasterframes...

I need to make data + nodata = data and not data + nodata> = nodata If understand the code rf_local_add consider data + nodata = nodata rf_local_biased_add consider data> + nodata> = data

anyone can help me to do this without this function (maybe masking etc)?

Thanks

lepitrust commented 2 years ago

Up... Anyone know why this feature is not available in pyrasterframes?

https://github.com/locationtech/rasterframes/blob/dc72ece3534a1d8d9a9eebfb1ce4b90e9c3f6ce8/core/src/main/scala/org/locationtech/rasterframes/expressions/localops/BiasedAdd.scala

Thanks

pomadchin commented 2 years ago

Hey @lepitrust mb it just was not registered and than not exposed; I'll be glad to help you with framing a PR in case you want to handle that yourself.

As a workaround you may try to reg this function and use CQL to call it.

mb there was also smth behind it cc @metasim

lepitrust commented 2 years ago

Hi, thanks @pomadchin! Any alternative is welcome ;) I have a little spark cluster with pyrasterframes correctly working. I'm trying to use for sum two raster after mul operation. beam and diff are raster tiles, k_b a k_d are float number.

This working, but diff raster contain a lot of NoData which must be ignored in the sum... The solution can be a biased op... ;)

df_rf_ghi = df_rf_max \ .select(rf_extent(F.col('beam')).alias('new_extent'), rf_crs(F.col('beam')).alias('new_crs'), \ F.col('dims').alias('ghi_dims') rf_local_add(\ rf_local_multiply(F.col('beam'), F.col('k_b')), \ rf_local_multiply(F.col('diff'), F.col('k_d')) \ ).alias('new_raster'))