dotnet / spark

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
https://dot.net/spark
MIT License
2.03k stars 315 forks source link

At least allow Spark 3.2.2 to be used #1122

Closed dragorosson closed 1 year ago

dragorosson commented 2 years ago

3.2.2 has been out for almost 4 months, and 2 months ago @Niharikadutta mentioned that support for it is being worked on here. What's the status of this? GCP Dataproc Serverless now cannot be run with Spark 3.2.1 (See Dataproc Serverless Spark runtime releases).

I'm not very familiar with this codebase so I don't really expect this PR to merge as-is, but I can get a basic Spark program to run on 3.2.2 by simply allowing it without any other changes...

Fixes #1095

dragorosson commented 1 year ago

@AFFogarty @Niharikadutta Should the failed checks be re-run?