taylorwood / clj.native-image

Build GraalVM native images with Clojure Deps and CLI tools
MIT License
271 stars 20 forks source link

FileNotFoundException: Could not locate <my class>.class #3

Closed aviflax closed 6 years ago

aviflax commented 6 years ago

This is probably PEBKAC as usual, but maybe you wouldn’t mind doing my debugging for me?

I thought it’d be helpful to have a fast executable for cljfmt so I could run it as a git pre-commit hook.

cljfmt itself doesn’t use tools.deps, but cljfmt-runner does, so I thought I’d see if I could use this project to build a native executable of that project.

Unfortunately after adding the native-image profile and the jvm-opts to deps.edn and (:gen-class) to the ns forms, I’m just getting this:

cljfmt-runner $ clojure -A:native-image-check
Loading cljfmt-runner.check
Exception in thread "main" java.io.FileNotFoundException: Could not locate cljfmt-runner/check__init.class, cljfmt-runner/check.clj or cljfmt-runner/check.cljc on classpath.
    at clojure.lang.RT.load(RT.java:464)
    at clojure.lang.RT.load(RT.java:426)
    at clojure.core$load$fn__6735.invoke(core.clj:6098)
    at clojure.core$load.invokeStatic(core.clj:6097)
    at clojure.core$load.doInvoke(core.clj:6081)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clj.native_image$_main.invokeStatic(native_image.clj:60)
    at clj.native_image$_main.doInvoke(native_image.clj:50)
        ...

Here’s the new stuff I added to deps.edn:

Bottom of deps.edn ```edn :native-image-check {:main-opts ["-m clj.native-image cljfmt-runner.check" "-H:Name=cljfmt-check"] :extra-deps {clj.native-image {:git/url "https://github.com/taylorwood/clj.native-image.git" :sha "2dd7ce300e152d56a65dfc28de7d860981bf85cf"}}}} :jvm-opts ["-Dclojure.compiler.direct-linking=true"]} ```


And:

ns form from src/cljfmt_runner/check.clj ```clojure (ns cljfmt-runner.check (:require [cljfmt.core :as cljfmt] [cljfmt-runner.core :refer :all] [cljfmt-runner.diff :as diff]) ;; Required for compilation to a “native image” executable via GraalVM. (:gen-class)) ```


If you’d like, you can check out my branch to reproduce.

Thanks!

taylorwood commented 6 years ago

Thanks for detailed report and repro case again! I’ll try to dig in this week and see what’s happening. Looks like it’s failing before the native image generation so it may well be a bug in my code (which is probably too naive about compilation at the moment).

aviflax commented 6 years ago

My pleasure! Sounds great.

BTW, I have a feeling it might have something to do with the hyphen in the namespace name, and how it translates to file paths — i.e. the convention that Clojure namespaces use hyphens rather than underscores but in the names of files and directories those hyphens are translated to underscores. I tried changing the hyphen in the class name argument in the profile to an underscore and got a different result… so, that might mean something.

taylorwood commented 6 years ago

You're right, there's an issue with class name munging that I handle in my Leiningen plugin but neglected in this one.

Once that bug is fixed, we hit another snag in native-image I mentioned in this other issue https://github.com/taylorwood/clj.native-image/issues/2#issuecomment-431239245 re: "unbalanced monitors" in clojure.core

error: unbalanced monitors: mismatch at monitorexit, 96|LoadField#lockee__5436__auto__ != 3|LoadField#lockee__5436__auto__

I worked around this issue by changing your fork's deps.edn to use Clojure 1.8 😛

Then I added --report-unsupported-elements-at-runtime to deps.edn in [:native-image :main-opts] to make easier to get a build, which resulted in a native image that immediately throws an exception when executed:

Exception in thread "main" java.lang.IllegalArgumentException: No matching field found: isFile for class java.io.File

We can use Graal's reflection config to address this. I made a reflection.json file (that almost certainly includes more elements than necessary):

[
    {
        "name": "java.io.File",
        "allDeclaredConstructors": true,
        "allPublicConstructors": true,
        "allDeclaredMethods": true,
        "allPublicMethods": true,
        "allPublicFields": true
    }
]

And reference that file from the deps.edn in the same [:native-image :main-opts] map as above with the following argument:

-H:ReflectionConfigurationFiles=reflection.json

Rinse, repeat, and now we get a native image that actually works! I introduced some formatting errors in the cljfmt-runner project itself, and now we have snappy native Clojure linter:

➜  cljfmt-runner git:(native-image) ✗ ./cljfmt-check
Checked 6 file(s)
2 file(s) were incorrectly formatted
--- a/deps.edn
+++ b/deps.edn
@@ -17,12 +17,12 @@
   :lint/fix {:main-opts ["-m" "cljfmt-runner.fix"]}

   :native-image {:main-opts   ["-m clj.native-image cljfmt-runner.check"
-                                "--report-unsupported-elements-at-runtime"
+                               "--report-unsupported-elements-at-runtime"
                                "-H:Name=cljfmt-check"
                                "-H:ReflectionConfigurationFiles=reflection.json"]
-                       :extra-deps  {clj.native-image
-                    {:local/root "../clj.native-image"}
-                                     #_{:git/url "https://github.com/taylorwood/clj.native-image.git"
-                                      :sha "2dd7ce300e152d56a65dfc28de7d860981bf85cf"}}}}
+                 :extra-deps  {clj.native-image
+                               {:local/root "../clj.native-image"}
+                               #_{:git/url "https://github.com/taylorwood/clj.native-image.git"
+                                  :sha "2dd7ce300e152d56a65dfc28de7d860981bf85cf"}}}}

  :jvm-opts ["-Dclojure.compiler.direct-linking=true"]}
--- a/src/cljfmt_runner/core.clj
+++ b/src/cljfmt_runner/core.clj
@@ -8,7 +8,7 @@
 (defn file?
   "Determine if the given java.io.File is a file (as opposed to a directory)."
   [file]
-    (.isFile file))
+  (.isFile file))

 (defn directory?
   "Determine if the given java.io.File is a directory."
➜  cljfmt-runner git:(native-image) ✗ /usr/bin/time -l ./cljfmt-check
<removed cljfmt output>
        0.21 real         0.16 user         0.04 sys
 124846080  maximum resident set size
<removed extra time info>
aviflax commented 6 years ago

Wow, great stuff, thanks again!!!!!