java.sql.Timestamp not guaranteed to exist

[weavejester]

In later versions of the JDK (from 11 up I believe), certain classes are not guaranteed to be available, including java.sql.Timestamp. This causes an issue with fipp.ednize:

REPL-y 0.4.3, nREPL 0.6.0
Clojure 1.10.0
OpenJDK 64-Bit Server VM 13.0.1+9
...
user=> (require 'fipp.ednize)
Syntax error (ClassNotFoundException) compiling at (ednize.clj:1:1).
java/sql/Timestamp

[brandonbloom]

Thanks for the heads up, I'm not running the latest JDK yet on my projects.

In general, I'm not quite sure what to do about optional dependencies and load order. Open to suggestions. This has come up before with Datomic's data types, where folks wanted pretty printing for those.

I think the simplest thing to do may be to pull out optional dependencies in to separate namespaces to be required independently, but the ergonomics of that are less than ideal.

[weavejester]

I've investigated this a little further and it only occurs in Leiningen REPLs that are started outside of a project, or inside a project marked as :eval-in-leiningen, which is curious.

I'll investigate further, but as a more general solution for things like Datomic's types, you should be able implement the protocol at runtime iff the class exists. There's an example of it in Ring if you want to take a look.

[vemv]

Hi,

I'm developing a Lein plugin, having released some preliminary versions.

The plugin depends on fipp latest (0.6.3).

We've noticed that the plugin fails due to this issue, if using a JDK > 8:

``` Syntax error compiling at (fipp/ednize.clj:1:1). at clojure.lang.Compiler.load(Compiler.java:7648) at clojure.lang.RT.loadResourceScript(RT.java:381) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.load(RT.java:459) at clojure.lang.RT.load(RT.java:424) at clojure.core$load$fn__6839.invoke(core.clj:6126) at clojure.core$load.invokeStatic(core.clj:6125) at clojure.core$load.doInvoke(core.clj:6109) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5908) at clojure.core$load_one.invoke(core.clj:5903) at clojure.core$load_lib$fn__6780.invoke(core.clj:5948) at clojure.core$load_lib.invokeStatic(core.clj:5947) at clojure.core$load_lib.doInvoke(core.clj:5928) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$load_libs.invokeStatic(core.clj:5985) at clojure.core$load_libs.doInvoke(core.clj:5969) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$require.invokeStatic(core.clj:6007) at clojure.core$require.doInvoke(core.clj:6007) at clojure.lang.RestFn.invoke(RestFn.java:421) at fipp.visit$eval711$loading__6721__auto____712.invoke(visit.cljc:1) at fipp.visit$eval711.invokeStatic(visit.cljc:1) at fipp.visit$eval711.invoke(visit.cljc:1) at clojure.lang.Compiler.eval(Compiler.java:7177) at clojure.lang.Compiler.eval(Compiler.java:7166) at clojure.lang.Compiler.load(Compiler.java:7636) at clojure.lang.RT.loadResourceScript(RT.java:381) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.load(RT.java:459) at clojure.lang.RT.load(RT.java:424) at clojure.core$load$fn__6839.invoke(core.clj:6126) at clojure.core$load.invokeStatic(core.clj:6125) at clojure.core$load.doInvoke(core.clj:6109) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5908) at clojure.core$load_one.invoke(core.clj:5903) at clojure.core$load_lib$fn__6780.invoke(core.clj:5948) at clojure.core$load_lib.invokeStatic(core.clj:5947) at clojure.core$load_lib.doInvoke(core.clj:5928) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$load_libs.invokeStatic(core.clj:5985) at clojure.core$load_libs.doInvoke(core.clj:5969) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$require.invokeStatic(core.clj:6007) at clojure.core$require.doInvoke(core.clj:6007) at clojure.lang.RestFn.invoke(RestFn.java:436) at fipp.clojure$eval703$loading__6721__auto____704.invoke(clojure.cljc:1) at fipp.clojure$eval703.invokeStatic(clojure.cljc:1) at fipp.clojure$eval703.invoke(clojure.cljc:1) at clojure.lang.Compiler.eval(Compiler.java:7177) at clojure.lang.Compiler.eval(Compiler.java:7166) at clojure.lang.Compiler.load(Compiler.java:7636) at clojure.lang.RT.loadResourceScript(RT.java:381) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.load(RT.java:459) at clojure.lang.RT.load(RT.java:424) at clojure.core$load$fn__6839.invoke(core.clj:6126) at clojure.core$load.invokeStatic(core.clj:6125) at clojure.core$load.doInvoke(core.clj:6109) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5908) at clojure.core$load_one.invoke(core.clj:5903) at clojure.core$load_lib$fn__6780.invoke(core.clj:5948) at clojure.core$load_lib.invokeStatic(core.clj:5947) at clojure.core$load_lib.doInvoke(core.clj:5928) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$load_libs.invokeStatic(core.clj:5985) at clojure.core$load_libs.doInvoke(core.clj:5969) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$require.invokeStatic(core.clj:6007) at clojure.core$require.doInvoke(core.clj:6007) at clojure.lang.RestFn.invoke(RestFn.java:551) at leiningen.resolve_java_sources_and_javadocs$eval695$loading__6721__auto____696.invoke(resolve_java_sources_and_javadocs.clj:1) at leiningen.resolve_java_sources_and_javadocs$eval695.invokeStatic(resolve_java_sources_and_javadocs.clj:1) at leiningen.resolve_java_sources_and_javadocs$eval695.invoke(resolve_java_sources_and_javadocs.clj:1) at clojure.lang.Compiler.eval(Compiler.java:7177) at clojure.lang.Compiler.eval(Compiler.java:7166) at clojure.lang.Compiler.load(Compiler.java:7636) at clojure.lang.RT.loadResourceScript(RT.java:381) at clojure.lang.RT.loadResourceScript(RT.java:372) at clojure.lang.RT.load(RT.java:459) at clojure.lang.RT.load(RT.java:424) at clojure.core$load$fn__6839.invoke(core.clj:6126) at clojure.core$load.invokeStatic(core.clj:6125) at clojure.core$load.doInvoke(core.clj:6109) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5908) at clojure.core$load_one.invoke(core.clj:5903) at clojure.core$load_lib$fn__6780.invoke(core.clj:5948) at clojure.core$load_lib.invokeStatic(core.clj:5947) at clojure.core$load_lib.doInvoke(core.clj:5928) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$load_libs.invokeStatic(core.clj:5985) at clojure.core$load_libs.doInvoke(core.clj:5969) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$require.invokeStatic(core.clj:6007) at clojure.core$require.doInvoke(core.clj:6007) at clojure.lang.RestFn.invoke(RestFn.java:408) at leiningen.core.utils$require_resolve.invokeStatic(utils.clj:102) at leiningen.core.utils$require_resolve.invoke(utils.clj:95) at leiningen.core.project$apply_middleware.invokeStatic(project.clj:853) at leiningen.core.project$apply_middleware.invoke(project.clj:846) at clojure.core.protocols$naive_seq_reduce.invokeStatic(protocols.clj:62) at clojure.core.protocols$interface_or_naive_reduce.invokeStatic(protocols.clj:72) at clojure.core.protocols$fn__8159.invokeStatic(protocols.clj:169) at clojure.core.protocols$fn__8159.invoke(protocols.clj:124) at clojure.core.protocols$fn__8114$G__8109__8123.invoke(protocols.clj:19) at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) at clojure.core.protocols$fn__8146.invokeStatic(protocols.clj:75) at clojure.core.protocols$fn__8146.invoke(protocols.clj:75) at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13) at clojure.core$reduce.invokeStatic(core.clj:6828) at clojure.core$reduce.invoke(core.clj:6810) at leiningen.core.project$apply_middleware.invokeStatic(project.clj:848) at leiningen.core.project$apply_middleware.invoke(project.clj:846) at leiningen.core.project$activate_middleware.invokeStatic(project.clj:880) at leiningen.core.project$activate_middleware.invoke(project.clj:876) at leiningen.update_in$update_project.invokeStatic(update_in.clj:22) at leiningen.update_in$update_project.invoke(update_in.clj:16) at leiningen.update_in$update_in.invokeStatic(update_in.clj:37) at leiningen.update_in$update_in.doInvoke(update_in.clj:24) at clojure.lang.RestFn.applyTo(RestFn.java:146) at clojure.lang.Var.applyTo(Var.java:705) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$partial_task$fn__7331.doInvoke(main.clj:284) at clojure.lang.RestFn.applyTo(RestFn.java:139) at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$apply_task.invokeStatic(main.clj:334) at leiningen.core.main$apply_task.invoke(main.clj:320) at leiningen.core.main$resolve_and_apply.invokeStatic(main.clj:343) at leiningen.core.main$resolve_and_apply.invoke(main.clj:336) at leiningen.update_in$update_in.invokeStatic(update_in.clj:37) at leiningen.update_in$update_in.doInvoke(update_in.clj:24) at clojure.lang.RestFn.applyTo(RestFn.java:146) at clojure.lang.Var.applyTo(Var.java:705) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$partial_task$fn__7331.doInvoke(main.clj:284) at clojure.lang.RestFn.applyTo(RestFn.java:139) at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$apply_task.invokeStatic(main.clj:334) at leiningen.core.main$apply_task.invoke(main.clj:320) at leiningen.core.main$resolve_and_apply.invokeStatic(main.clj:343) at leiningen.core.main$resolve_and_apply.invoke(main.clj:336) at leiningen.update_in$update_in.invokeStatic(update_in.clj:37) at leiningen.update_in$update_in.doInvoke(update_in.clj:24) at clojure.lang.RestFn.applyTo(RestFn.java:146) at clojure.lang.Var.applyTo(Var.java:705) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$partial_task$fn__7331.doInvoke(main.clj:284) at clojure.lang.RestFn.applyTo(RestFn.java:139) at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$apply_task.invokeStatic(main.clj:334) at leiningen.core.main$apply_task.invoke(main.clj:320) at leiningen.with_profile$with_profiles_STAR_.invokeStatic(with_profile.clj:14) at leiningen.with_profile$with_profiles_STAR_.invoke(with_profile.clj:8) at leiningen.with_profile$apply_task_with_profiles.invokeStatic(with_profile.clj:53) at leiningen.with_profile$apply_task_with_profiles.invoke(with_profile.clj:45) at leiningen.with_profile$with_profile$fn__11657.invoke(with_profile.clj:85) at clojure.core$mapv$fn__8445.invoke(core.clj:6912) at clojure.core.protocols$fn__8159.invokeStatic(protocols.clj:168) at clojure.core.protocols$fn__8159.invoke(protocols.clj:124) at clojure.core.protocols$fn__8114$G__8109__8123.invoke(protocols.clj:19) at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31) at clojure.core.protocols$fn__8146.invokeStatic(protocols.clj:75) at clojure.core.protocols$fn__8146.invoke(protocols.clj:75) at clojure.core.protocols$fn__8088$G__8083__8101.invoke(protocols.clj:13) at clojure.core$reduce.invokeStatic(core.clj:6828) at clojure.core$mapv.invokeStatic(core.clj:6903) at clojure.core$mapv.invoke(core.clj:6903) at leiningen.with_profile$with_profile.invokeStatic(with_profile.clj:85) at leiningen.with_profile$with_profile.doInvoke(with_profile.clj:63) at clojure.lang.RestFn.applyTo(RestFn.java:146) at clojure.lang.Var.applyTo(Var.java:705) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$partial_task$fn__7331.doInvoke(main.clj:284) at clojure.lang.RestFn.applyTo(RestFn.java:139) at clojure.lang.AFunction$1.doInvoke(AFunction.java:31) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$apply.invoke(core.clj:660) at leiningen.core.main$apply_task.invokeStatic(main.clj:334) at leiningen.core.main$apply_task.invoke(main.clj:320) at leiningen.core.main$resolve_and_apply.invokeStatic(main.clj:343) at leiningen.core.main$resolve_and_apply.invoke(main.clj:336) at leiningen.core.main$_main$fn__7420.invoke(main.clj:453) at leiningen.core.main$_main.invokeStatic(main.clj:442) at leiningen.core.main$_main.doInvoke(main.clj:439) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.lang.Var.applyTo(Var.java:705) at clojure.core$apply.invokeStatic(core.clj:665) at clojure.main$main_opt.invokeStatic(main.clj:514) at clojure.main$main_opt.invoke(main.clj:510) at clojure.main$main.invokeStatic(main.clj:664) at clojure.main$main.doInvoke(main.clj:616) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.lang.Var.applyTo(Var.java:705) at clojure.main.main(main.java:40) Caused by: java.lang.ExceptionInInitializerError at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:398) at clojure.lang.RT.classForName(RT.java:2211) at clojure.lang.RT.classForName(RT.java:2220) at clojure.lang.RT.loadClassForName(RT.java:2239) at clojure.lang.RT.load(RT.java:449) at clojure.lang.RT.load(RT.java:424) at clojure.core$load$fn__6839.invoke(core.clj:6126) at clojure.core$load.invokeStatic(core.clj:6125) at clojure.core$load.doInvoke(core.clj:6109) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.core$load_one.invokeStatic(core.clj:5908) at clojure.core$load_one.invoke(core.clj:5903) at clojure.core$load_lib$fn__6780.invoke(core.clj:5948) at clojure.core$load_lib.invokeStatic(core.clj:5947) at clojure.core$load_lib.doInvoke(core.clj:5928) at clojure.lang.RestFn.applyTo(RestFn.java:142) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$load_libs.invokeStatic(core.clj:5985) at clojure.core$load_libs.doInvoke(core.clj:5969) at clojure.lang.RestFn.applyTo(RestFn.java:137) at clojure.core$apply.invokeStatic(core.clj:667) at clojure.core$require.invokeStatic(core.clj:6007) at clojure.core$require.doInvoke(core.clj:6007) at clojure.lang.RestFn.invoke(RestFn.java:421) at fipp.ednize$eval746$loading__6721__auto____747.invoke(ednize.clj:1) at fipp.ednize$eval746.invokeStatic(ednize.clj:1) at fipp.ednize$eval746.invoke(ednize.clj:1) at clojure.lang.Compiler.eval(Compiler.java:7177) at clojure.lang.Compiler.eval(Compiler.java:7166) at clojure.lang.Compiler.load(Compiler.java:7636) ... 218 more Caused by: java.lang.ClassNotFoundException: java/sql/Timestamp at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:398) at clojure.lang.RT.classForName(RT.java:2211) at clojure.lang.RT.classForNameNonLoading(RT.java:2224) at clojure.instant$loading__6721__auto____8334.invoke(instant.clj:9) at clojure.instant__init.load(Unknown Source) at clojure.instant__init.(Unknown Source) ... 249 more ```

I'd be greatly thankful if the issue was solved. It should be a matter of splitting the https://github.com/brandonbloom/fipp/blob/3e29a69cd7b93310441efa1de66c98459384ee9c/src/fipp/ednize.clj#L33 into two extend-protocols, a large unconditional one for the non-problematic classes, and a conditional one, wrapped in something like (when (try (Class/forName "...") (catch ClassNotFoundException _ false))).

[brandonbloom]

Hey @vemv, I'd consider a PR that does as you suggest, but I'm concerned that the issue isn't whether or not the class is available. It's when the class is available. If you load Fipp and then later load the java.sql package, there is no mechanism to properly extend the protocol to that type. Or have I misunderstood the situation? That said, it might be preferable to degrade and solve the problem I'm highlighting later. I don't personally have this issue, so I'm open to suggestions from folks on the right way to proceed.

[vemv]

Thanks for the response! I'll try to determine the exact cause of the issue so that this can be confidently fixed.

...to the best of my knowledge, :import does "nothing" (java classes don't have to be loaded - they're always in the classpath, which is immutable) but it's better to examine those assumptions cautiously.

Perhaps it has something to do with Java's module system.

Edit: this is the relevant issue https://github.com/technomancy/leiningen/issues/2149

[vemv]

fipp's issue is basically an instance of https://github.com/technomancy/leiningen/issues/2149 :

Basically, Lein itself (but not Lein-calculated JVMs), including plugins, have a limited classpath (aka the bootclasspath), which starting from JDK9 does not include java.sql* classes.

Relevant quote from https://github.com/technomancy/leiningen/issues/2149:

I believe this is caused by java 9's module system hiding java.sql.* from the bootstrap classloader, and exposing it via the platform classloader instead (see http://openjdk.java.net/jeps/261, and search for "java.sql" to find the relevant paragraph).

---

From what I know, nothing in Clojure code can fix this situation - it can only be detected. (This explains why neither Lein or clojure.core do an attempt at "force-loading" java.sql.Timestamp somehow).

Checking the source for clojure.core/import, one can verify that :import boils down to a class to Class/forName with the false argument (i.e. non-loading: one will use JVM's classloaders instead of Clojure's, and no class initializers will be run)

https://github.com/clojure/clojure/blob/0df3d8e2e27fb06fa53398754cac2be4878b12d1/src/jvm/clojure/lang/Compiler.java#L777-L778 https://github.com/clojure/clojure/blob/0df3d8e2e27fb06fa53398754cac2be4878b12d1/src/jvm/clojure/lang/RT.java#L2224-L2225

tldr, from what I can tell (when (try (Class/forName "...") (catch ClassNotFoundException _ false))) is harmless - it merely removes a feature when it's not possibly available.

[brandonbloom]

OK, then PR welcome.

[camsaul]

Running into this with Java 14 as well

[vemv]

Running into this with Java 14 as well

Opened https://github.com/brandonbloom/fipp/pull/78 now. This fix has worked well for me over the last few months.