Open sola1tmy opened 3 weeks ago
Hi! Thanks for your interest :) I'd say, if you can use the latest version of Spark (3.3) supported by the Kotlin API in your project, you could at least try if it works for you. The plan for 3.4+ is to keep the migration as small as possible.
I'm not yet sure whether the Kotlin Spark API should move to spark-connect only or also generic spark, but if you keep your project to the concepts mentioned here you should be good to go for any direction.
The main thing to take away is to try to stick to your own defined data classes for Datasets (so try to avoid Pair, Triple, and data classes from other libraries). This will make the eventual transition easier as for 3.4, encodable Kotlin data classes will have to be annotation with @Sparkify
.
We are consider integration some spark related functions in our project. But seems the progress for support spark 3.4 in this project still have a lot of work.
https://github.com/Kotlin/kotlin-spark-api/issues/195
we consider
So should we use this project now? when will the next release come?And will the api keep same next release?