locationtech / rasterframes

Geospatial Raster support for Spark DataFrames
http://rasterframes.io
Apache License 2.0
240 stars 46 forks source link

Spark 3.4 #611

Closed thomas-maschler closed 9 months ago

thomas-maschler commented 1 year ago

Add support for Spark 3.2.4, 3.3.2 and 3.4.0

pomadchin commented 1 year ago

I'll also publish https://github.com/locationtech/geotrellis/pull/3513 once CQs are cleared!

thomas-maschler commented 1 year ago

@pomadchin Rasterframes is still on geotrellis 3.6.3. There were some dependency conflicts which prevented us from updating. I will see if it builds with the latest release

pomadchin commented 1 year ago

@thomas-maschler oh, I guess it's stac4s?

thomas-maschler commented 1 year ago

yes, I get

[error]     * org.typelevel:cats-effect_2.12:3.4.8 (early-semver) is selected over {3.4.6, 2.5.4}
[error]         +- org.locationtech.geotrellis:geotrellis-store_2.12:3.7.0 (depends on 3.4.8)
[error]         +- com.softwaremill.sttp.client3:catsce2_2.12:3.7.0   (depends on 2.5.4)
[error]         +- co.fs2:fs2-core_2.12:3.6.1                         (depends on 3.4.6)
[error]
[error]     * co.fs2:fs2-core_2.12:3.6.1 (early-semver) is selected over 2.5.11
[error]         +- co.fs2:fs2-io_2.12:3.6.1                           (depends on 3.6.1)
[error]         +- org.locationtech.geotrellis:geotrellis-store_2.12:3.7.0 (depends on 3.6.1)
[error]         +- com.azavea.stac4s:client_2.12:0.8.1                (depends on 2.5.11)
thomas-maschler commented 1 year ago

@echeipesh, the publishing CI workflow is still incomplete and only builds jar and wheel on the worker but does not publish the files to Maven or Pypi. Do you want to keep this manual for now?

pomadchin commented 1 year ago

@thomas-maschler stac4s migrated into the stac org! https://github.com/stac-utils/stac4s

I'll update it once I have some time (:

pomadchin commented 9 months ago

🔥

pomadchin commented 9 months ago

Let's merge it as is for now!