Focal Operations and Buffer Tile support

locationtech / rasterframes

Geospatial Raster support for Spark DataFrames

http://rasterframes.io

Apache License 2.0

240 stars 46 forks source link

Focal Operations and Buffer Tile support #563

Closed pomadchin closed 2 years ago

pomadchin commented 2 years ago

It is a port of https://github.com/locationtech/rasterframes/pull/551

TODO:

[x] Focal Expressions should be of a correct arity
[x] Python Focal API ~, blocked by https://github.com/locationtech/rasterframes/pull/562~
- Docker image for tests: quay.io/daunnc/rasterframes-notebook:0.9.2-SNAPSHOT
[x] Python STAC API
[x] After reading a buffered tile and performing a local operation the buffer is lost: I propose that to be not a part of this PR, but to move it into a separate one as an enchancement.

Focal Operations notebook screenshot

STAC API Example notebook screenshot

Closes #441 Closes #551 Closes #487

Blocked by https://github.com/locationtech/geotrellis/pull/3419

metasim commented 2 years ago

@pomadchin I'm not sure I understand what you mean by these two sentenses:

All functions can't be used in spark-sql. All Focal operations are unary and can't be applied to two expressions as that can be expected by the user.

pomadchin commented 2 years ago

@metasim Do we want to allow users to write cql expressions:

select rf_hillshade(tile, 400, 10, 1)

Right now Focal ops are not allowing that, should not Hillshade ops in the case be QuaternaryExpression to parse all arguments?

metasim commented 2 years ago

@metasim Do we want to allow users to write cql expressions:
select rf_hillshade(tile, 400, 10, 1)
Right now Focal ops are not allowing that, should not Hillshade ops in the case be QuaternaryExpression to parse all arguments?

Yes, eventually... I try to maintain Scala, Python and SQL parity. I'll look more deeply on what the issue is and what the workarounds might be, because I don't ever remember this being an issue (if you have the function as an Expression, it works in all the contexts).