Scala 2.12 and Spark 3 support

echeipesh commented 3 years ago

This PR bumps to Spark 3 and Scala 2.12, dropping support for Scala 2.11

Two major problems faced by this PR as:

ExpresisonEncoder now has expectations for what serializer and deserializer expressions look like and rewrites them on construction. Combined with getting rid of flat property this has introduced inconsistency in how RF encoders are treated when they're top of the column vs when they're member of a struct.
Filter is now sealed class. It's no longer possible to push down spatial filter predicates to GeoTrellis relation. This is sad but probably not a huge deal in reality. Very much secondary issue.

pomadchin commented 2 years ago

@echeipesh @metasim How about merging it as is without fixing the Python for now to unblock #551?

metasim commented 2 years ago

@echeipesh @metasim How about merging it as is without fixing the Python for now to unblock #551?

Let's go for it! I'll work the python fixes on another PR.

metasim commented 2 years ago

@echeipesh @pomadchin Congratulations!! 🎉 Amazing work!!

pomadchin commented 2 years ago

Closes #530 as well!

locationtech / rasterframes