kscripting / kscript

Scripting enhancements for Kotlin
MIT License
2.07k stars 125 forks source link

Unable to access java.sql namespace #163

Open ayvazj opened 6 years ago

ayvazj commented 6 years ago

Unable to access redshift database using this as a guide. https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-in-code.html

Given the errors are ClassNotFoundException I thought it might be a configuration issue.

Exception in thread "main" java.lang.NoClassDefFoundError: java/sql/SQLException
    at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
    at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3139)
    at java.base/java.lang.Class.getDeclaredMethod(Class.java:2430)
    at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:46)
    at org.jetbrains.kotlin.runner.Main.run(Main.kt:109)
    at org.jetbrains.kotlin.runner.Main.main(Main.kt:119)
Caused by: java.lang.ClassNotFoundException: java.sql.SQLException
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:466)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:563)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
    ... 6 more
holgerbrandl commented 6 years ago

Could you post a more self-contained example (including dependency declarations)?

I could imagine that it is just another flavor of https://github.com/holgerbrandl/kscript/issues/155#issuecomment-419034634 See also https://github.com/holgerbrandl/kscript/issues/109

ayvazj commented 6 years ago
#!/usr/bin/env kscript
// See here for kscript installation instructions
// https://github.com/holgerbrandl/kscript

@file:DependsOn("com.amazon.redshift:redshift-jdbc4:1.1.17.1017")
@file:MavenRepository("redshift", "http://redshift-maven-repository.s3-website-us-east-1.amazonaws.com/release")
@file:CompilerOpts("-jvm-target 1.8")

import java.sql.*
import java.util.*
import java.util.concurrent.TimeUnit

fun main(args: Array<String>) {
    var conn: Connection? = null
    var stmt: Statement? = null
    var rs: ResultSet? = null
    val dbURL = "***jdbc cluster connection string ****"
    val MasterUsername = "***master user name***"
    val MasterUserPassword = "***master user password***"
    try {
        //Dynamically load driver at runtime.
        //Redshift JDBC 4.1 driver: com.amazon.redshift.jdbc41.Driver
        //Redshift JDBC 4 driver: com.amazon.redshift.jdbc4.Driver
        Class.forName("com.amazon.redshift.jdbc42.Driver")

        //Open a connection and define properties.
        println("Connecting to database...")
        val props = Properties()

        //Uncomment the following line if using a keystore.
        //props.setProperty("ssl", "true");
        props.setProperty("user", MasterUsername)
        props.setProperty("password", MasterUserPassword)
        conn = DriverManager.getConnection(dbURL, props)

        //Try a simple query.
        println("Listing system tables...")
        stmt = conn.createStatement()
        val sql = "select * from information_schema.tables;"
        rs = stmt.executeQuery(sql)

        //Get the data from the result set.
        while (rs.next()) {
            //Retrieve two columns.
            val catalog = rs.getString("table_catalog")
            val name = rs.getString("table_name")

            //Display values.
            print("Catalog: ${catalog}")
            println(", Name: ${name}")
        }
    } catch (e: Exception) {
        //For convenience, handle all errors here.
        e.printStackTrace()
    } finally {
        //Finally block to close resources.
        try {
            stmt?.close()
            rs?.close()
            conn?.close()
        } catch (e: SQLException) {
        }
    }
    println("Finished connectivity test.")
}
holgerbrandl commented 6 years ago

When running exactly the same bits with

        Class.forName("com.amazon.redshift.jdbc4.Driver")

instead, I get:

Connecting to database...
java.util.NoSuchElementException
    at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
    at com.amazon.jdbc.common.BaseConnectionFactory.acceptsURL(Unknown Source)
    at com.amazon.jdbc.common.AbstractDriver.connect(Unknown Source)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:208)
    at Kscript_sql_ticket163Kt.main(kscript_sql_ticket163.kt:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:61)
    at org.jetbrains.kotlin.runner.Main.run(Main.kt:109)
    at org.jetbrains.kotlin.runner.Main.main(Main.kt:119)
java.sql.SQLException: No suitable driver found for ***jdbc cluster connection string ****
    at java.sql.DriverManager.getConnection(DriverManager.java:689)
    at java.sql.DriverManager.getConnection(DriverManager.java:208)
    at Kscript_sql_ticket163Kt.main(kscript_sql_ticket163.kt:35)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:61)
    at org.jetbrains.kotlin.runner.Main.run(Main.kt:109)
    at org.jetbrains.kotlin.runner.Main.main(Main.kt:119)

which is what I'd expect to see given that I don't know pw or user. I'm on

holgerbrandl commented 5 years ago

Closed due to inactivity. Feel welcome to keep commenting on this thread/

gianluz commented 5 years ago

I had a similar problem using a personal library that uses Gson inside

java.lang.NoClassDefFoundError: java/sql/Time

        at com.google.gson.Gson.<

looks like the java.sql.* JDBC stuff is not included when we run the script.

holgerbrandl commented 5 years ago

@gianluz Could you provide a complete example?

gloeglm commented 5 years ago

I think this might be related to the java module system. I can run this small script:

#!/usr/bin/env kscript

//DEPS com.vladsch.kotlin-jdbc:kotlin-jdbc:0.4.8,mysql:mysql-connector-java:5.1.47

import com.vladsch.kotlin.jdbc.*

Class.forName("com.mysql.jdbc.Driver")
val session = session("jdbc:mysql://xxx", "xxx", "xxx")
val query = sqlQuery("select 1 as val")
val ids: List<Int> = session.list(query) { row -> row.int("val") }
println(ids)

fine with JAVA_HOME set to a java 8 vm, but with openjdk11 I get a java.lang.ClassNotFoundException: java.sql.Driver error.

I tried to use //KOTLIN_OPTS in various ways to maybe get the java.sql module enabled, but no luck so far

holgerbrandl commented 5 years ago

Indeed the annoying module system finally found its way into kscript. :-|

Just as a reminder for me

/usr/libexec/java_home -V
// export JAVA_HOME=`/usr/libexec/java_home -v 10`
// export JAVA_HOME=`/usr/libexec/java_home -v 1.8`

@gloeglm How would you normally configure the module system when calling java to allow access to java.sql?

gloeglm commented 5 years ago

@holgerbrandl
Usually the java.sql module should be enabled by default actually. In a normal JVM you can list the enabled modules by running java --list-modules - I am not sure where that module gets lost.

On a plain JVM re-enabling certain modules works by passing --add-modules java.sql to as a JVM argument, at least thats how it works for f.e. re-enabling jaxb

gloeglm commented 5 years ago

I figured out the command thats actually being run by adding set +x to the kotlinc script on my system, and it looks something like this:

/usr/lib/jvm/java-11-openjdk-amd64/bin/java -Xmx256M -Xms32M -Xmx4g -Dkotlin.home=/home/m.gloegl/.sdkman/candidates/kotlin/1.2.41 -cp /home/m.gloegl/.sdkman/candidates/kotlin/1.2.41/lib/kotlin-runner.jar org.jetbrains.kotlin.runner.Main -classpath /home/m.gloegl/.kscript/my_script.ef4ca496ee5b37ea.jar:/home/m.gloegl/.sdkman/candidates/kotlin/1.2.41/lib/kotlin-script-runtime.jar:/home/m.gloegl/.m2/repository/com/vladsch/kotlin-jdbc/kotlin-jdbc/0.4.8/kotlin-jdbc-0.4.8.jar:/home/m.gloegl/.m2/repository/org/jetbrains/kotlin/kotlin-reflect/1.2.61/kotlin-reflect-1.2.61.jar:/home/m.gloegl/.m2/repository/com/zaxxer/HikariCP/3.1.0/HikariCP-3.1.0.jar:/home/m.gloegl/.m2/repository/org/jetbrains/annotations/15.0/annotations-15.0.jar:/home/m.gloegl/.m2/repository/com/vladsch/boxed-json/boxed-json/0.5.16/boxed-json-0.5.16.jar:/home/m.gloegl/.m2/repository/org/glassfish/javax.json/1.0.4/javax.json-1.0.4.jar:/home/m.gloegl/.m2/repository/joda-time/joda-time/2.9.9/joda-time-2.9.9.jar:/home/m.gloegl/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar:/home/m.gloegl/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib-common/1.2.61/kotlin-stdlib-common-1.2.61.jar:/home/m.gloegl/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib-jdk8/1.2.61/kotlin-stdlib-jdk8-1.2.61.jar:/home/m.gloegl/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib-jdk7/1.2.61/kotlin-stdlib-jdk7-1.2.61.jar:/home/m.gloegl/.m2/repository/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar:/home/m.gloegl/.m2/repository/com/google/code/gson/gson/2.8.5/gson-2.8.5.jar:/home/m.gloegl/.m2/repository/com/fatboyindustrial/gson-javatime-serialisers/gson-javatime-serialisers/1.1.1/gson-javatime-serialisers-1.1.1.jar:/home/m.gloegl/.m2/repository/com/beust/klaxon/3.0.1/klaxon-3.0.1.jar:/home/m.gloegl/.m2/repository/org/jetbrains/kotlin/kotlin-stdlib/1.2.10/kotlin-stdlib-1.2.10.jar Main_My_script

Adding -list-modules to the java command shows the java.sql module as loaded. Might this be due to the additional classloader wrapping stuff being done in https://github.com/JetBrains/kotlin/blob/master/compiler/cli/cli-runner/src/org/jetbrains/kotlin/runner/runners.kt ? The URLClassLoader that is used to load the script main class is explicitly constructed with a null parent.

PS: Gave updating kotlin/kscript to latest version a go too, error still is the same

gloeglm commented 5 years ago

Refer to: http://java9.wtf/class-loading/ - I think this describes the root issue in the kotlin runner. You can demonstrate that by trying to load the class in the REPL:

Welcome to Kotlin version 1.3.10 (JRE 10.0.2+13-Ubuntu-1ubuntu0.18.04.3)
Type :help for help, :quit for quit
>>> Class.forName("java.sql.Driver")
java.lang.ClassNotFoundException: java.sql.Driver
    at java.base/java.lang.ClassLoader.findClass(ClassLoader.java:711)
    at org.jetbrains.kotlin.cli.common.repl.ReplClassLoader.findClass(ReplClassLoader.java:44)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:566)
gloeglm commented 5 years ago

Reported this as a kotlin issue: https://youtrack.jetbrains.com/issue/KT-28475

holgerbrandl commented 5 years ago

Thanks for reporting it to jetbrains. Indeed the classloader of kotlin has some issues. Fingers crossed that they will fix is soon. Classloader stuff is one of the most often reported kscript issues (in various flavors of course).

alexproca commented 5 years ago

I'm using this as a workaround on osX. Downgrading to java 1.8 before executing script.

#!/usr/bin/env bash
JAVA_HOME=`/usr/libexec/java_home -v 1.8`
// 2> /dev/null; cat <<EOF | kscript -

//DEPS org.jetbrains.exposed:exposed:0.11.2
//DEPS org.xerial:sqlite-jdbc:3.21.0.1
import org.jetbrains.exposed.sql.*

Database.connect("jdbc:sqlite:data.db", driver = "org.sqlite.JDBC")

println("Hello from Kotlin!")
for (arg in args) {
    println("arg: $arg")
}

EOF
holgerbrandl commented 5 years ago

That's a very interesting workaround. Thanks for sharing @alexproca.

holgerbrandl commented 4 years ago

Not strictly a duplicate, but centering around the same problem is #239

mukeshahirwar commented 4 years ago

Hello all,

please help me out , i am getting below error while connecting AS400(iseries server) from java , i am calling RPGLE Program.

Exception in thread "main" java.lang.NoClassDefFoundError: java/sql/Driver at java.base/java.lang.ClassLoader.defineClass1(Native Method) at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016) at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1095) i would be great help thansk

ScottPeterJohnson commented 3 years ago

I can confirm this still affects the 3.0 line. Workaround for the moment is just to package and then run the script.

holgerbrandl commented 3 years ago

I think this was to be expected since it is a kotin platform bug https://youtrack.jetbrains.com/issue/KT-28475

neupsh commented 3 years ago

hello, it looks like the kotlin bug is resolved but I still get NoClassDefFoundError: java/sql/Driver. Weirdly enough, it works if I run with kscript --interactive h2.kts and paste the code directly in the repl.

This does work on java 8, but not on 11+ (which is what I need to run one of my scripts). Any update on this issue?

h2.kts

#!/usr/bin/env kscript
@file:DependsOn("com.h2database:h2:1.4.200")

import java.util.*
import java.sql.*

var conn: Connection? = null
try {
    Class.forName("org.h2.Driver")
    conn = DriverManager.getConnection("jdbc:h2:mem:dev")
    val stmt = conn!!.createStatement()
    val rs = stmt.executeQuery("SHOW DATABASES;")
    while (rs.next()) {
        println(rs.getString(1))
    }
} catch (e: Exception) {
    e.printStackTrace()
} finally {
    conn?.close()
}
BoD commented 3 years ago

I, too, am stumbling upon this issue, and it's quite unfortunate since I was keen on using Gson (a fairly popular library)!

The same script works with standard kotlin script (main.kts), but I would prefer to use kscript for its packaging feature.

holgerbrandl commented 3 years ago

I strongly believe that this is still a platform bug that has been addressed just to some extent in the fix of https://youtrack.jetbrains.com/issue/KT-28475 for kotlinc but not yet for kotlin.

I've created another ticket describing this problem in https://youtrack.jetbrains.com/issue/KT-46312 . I think there is nothing we can do in kscript until this issue has been addressed.

MagnusMG commented 2 years ago

I'm not entirely sure that my problem is the same as the one discussed in this issue, but I have been getting java.lang.NoClassDefFoundError: java.sql.SQLException everytime I have run any kscript without first setting my jdk to 1.8. (Otherwise I use a later java version.) And I always forget to set the jdk, and then receive an error message that does not indicate what the solution is.

I thought this just had to do with Kotlin requiring 1.8, but after reading this thread (and trying out a few things) I realise that it has more to do with the fact that I always import a (home written) package that requires the java sql-library.

Anyway, based on @alexproca's workaround I wrote a shell script that first sets the jdk to 1.8 and then calls kscript:

#! /usr/bin/env zsh

export PATH=$(echo $PATH | sed -E -e "s;:'/System/Library/Frameworks/JavaVM.framework/Home/bin';;" -e "s;'/System/Library/Frameworks/JavaVM.framework/Home/bin':?;;") 
if [ -n "${JAVA_HOME+x}" ]; then
        export PATH=$(echo $PATH | sed -E -e "s;:${JAVA_HOME};;" -e "s;${JAVA_HOME}:?;;") 
fi
unset JAVA_HOME
export JAVA_HOME=`/usr/libexec/java_home -v 1.8`
export PATH=$JAVA_HOME/bin:$PATH

exec "/usr/local/bin/kscript" "$@" 

I saved this in a file called kscriptwithjava8, and used that one in my shebang at the top of my kt/kts file:

#! /usr/bin/env kscriptwithjava8
//Normal kscript file
...
alexproca commented 2 years ago

That's much more elegant than my workaround! Thanks for sharing 😊

Pyeroh commented 1 year ago

Any update on that ? I can't use Java 8 for my script, the dependencies I use are not compatible with Java 8, and kscript throw me a Unrecognized option: --add-modukes java.sql error. All Youtrack tickets are marked as fixed, I'm using kscript 4.1.1 with kotlin 1.7.21, and Java 17 on Linux. All should be well working, but it's still not...

andreas-mausch commented 1 year ago

I had this issue today and did some testing.

Java with a SQLException works as expected:

// javac A.java
// java A

import java.sql.SQLException;

public class A {
  public static void main (String[] args) throws SQLException {
    System.out.println("Hello World!");
    throw new SQLException();
    // throws as excepted: Exception in thread "main" java.sql.SQLException
  }
}

Kotlin (1.7.21) does not:

// kotlinc A.kt
// kotlin AKt.class

import java.sql.SQLException;

fun main() {
  println("Hello, World!")
  throw SQLException()
  // does not throw, but fails before:
  // Caused by: java.lang.ClassNotFoundException: java.sql.SQLException
}

I tried adding modules in a lot of ways (I'm not even sure about the syntax) but that didn't change anything: kotlin -J'--add-modules' -J'java.sql' AKt.class

@Pyeroh

All Youtrack tickets are marked as fixed

The important one (https://youtrack.jetbrains.com/issue/KT-46312) has Kotlin 1.8.0-Beta as target version, so I guess we have to wait for a new Kotlin release.

Edit: Apparently it has been released just 13h ago: https://github.com/JetBrains/kotlin/releases/tag/v1.8.0 Second Edit: I can confirm Kotlin 1.8.0 fixes the issue for my example and also for the kscript command!

Pyeroh commented 1 year ago

Can confirm this too, both run with kscript and packaged, I don't have errors anymore ! Thanks for the feedback on that !!