Jupyter integration not compatible with Java 8

zenwan commented 1 year ago

Java 8 still dominates, especially among industrial companies，

first executed at 2023.07.21 11:56:54 , Failed

retry at 2023.07.21 11:58 , Succeed, Each first run fails, and then retry succeeds

Related code： use spark(spark=3.2.0, scala=2.13, v=1.2.3） My Java version: java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1.8.0_121-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) kotlin-jupyter-kernel kotlin-jupyter-kernel 0.11.0.385

Jolanrensen commented 1 year ago

The Kotlin Spark API targets Java 8 everywhere except for the Jupyter integration at the moment. This was because the Kotlin Jupyter API required Java 11. I'll investigate whether Java 8 for Jupyter has been made possible since the last time I checked.

Jolanrensen commented 1 year ago

I updated the project with Jupyter integration at Java 8. The dev version is released to GitHub packages: https://github.com/Kotlin/kotlin-spark-api/packages/1527629. To use a package from GH packages in a notebook, you'll need to add it as a repository:

USE {
    repositories {
        maven {
            url = "https://maven.pkg.github.com/Kotlin/kotlin-spark-api"
            credentials {
                // Your GitHub user name and personal (classic) access token (https://github.com/settings/tokens)
                username = "zenwan"
                password = ""
            }
        }
    }

    // dependencies can be put here or in @file:DependsOn
    dependencies {
        implementation("org.apache.spark:spark-yarn_2.13:3.2.0")
    }
}

And in a new cell:

%use spark(spark=3.2.0, scala=2.13, v=1.2.4-SNAPSHOT, ...)

Could you give it a try?

Kotlin / kotlin-spark-api

Jupyter integration not compatible with Java 8 #199