search

clojure-emacs / cider-nrepl

A collection of nREPL middleware to enhance Clojure editors with common functionality like definition lookup, code completion, etc.
https://docs.cider.mx/cider-nrepl
687 stars 176 forks source link

Cider-macroexpand-all in deps.edn project of definterface crashes #721

Closed jellelicht closed 2 years ago

jellelicht commented 3 years ago

Expected behavior

Being able to macro expand forms that happen to define interfaces, in either deps.edn or leiningen projects

Actual behavior

A crash pop-up when macro expanding forms that happen to define interfaces, but only in deps.edn projects.

Steps to reproduce the problem

I made a tiny example that seems to be reproducible for at least one other person: https://github.com/jellelicht/cider-macro-repro

Steps to reproduce the problem (alt)

(macroexpand-1 '(definterface IFoo)) with cider-nrepl's machinery

Environment & Version information

CIDER version information

; Connected to nREPL server - nrepl://localhost:44805                                                                                 
;; CIDER 1.1.1 (Plovdiv), nREPL 0.8.3                                                                                                  
;; Clojure 1.10.3, Java 12

Emacs version

27.2

Operating system

GNU Guix (current master)

Stacktrace

``` Show: Project-Only All Hide: Clojure Java REPL Tooling Duplicates (52 frames hidden) This is an unexpected CIDER middleware error. Please submit a bug report via `M-x cider-report-bug`. If these stacktraces are occurring frequently, consider using the button(s) below to suppress these types of errors for the duration of your current CIDER session. The stacktrace buffer will still be generated, but it will "pop under" your current buffer instead of "popping over". The button toggles this behavior. Suppress macroexpand-error 2. Unhandled java.lang.ClassCastException class clojure.lang.Var$Unbound cannot be cast to class clojure.lang.DynamicClassLoader (clojure.lang.Var$Unbound and clojure.lang.DynamicClassLoader are in unnamed module of loader 'app') genclass.clj: 722 clojure.core/gen-interface genclass.clj: 688 clojure.core/gen-interface RestFn.java: 142 clojure.lang.RestFn/applyTo Var.java: 705 clojure.lang.Var/applyTo Compiler.java: 6997 clojure.lang.Compiler/macroexpand1 core.clj: 4012 clojure.core/macroexpand-1 core.clj: 4014 clojure.core/macroexpand walk.clj: 130 clojure.walk/macroexpand-all/fn walk.clj: 65 clojure.walk/prewalk walk.clj: 61 clojure.walk/prewalk core.clj: 2628 clojure.core/partial/fn core.clj: 2759 clojure.core/map/fn LazySeq.java: 42 clojure.lang.LazySeq/sval LazySeq.java: 51 clojure.lang.LazySeq/seq Cons.java: 39 clojure.lang.Cons/next RT.java: 713 clojure.lang.RT/next core.clj: 64 clojure.core/next core.clj: 3130 clojure.core/dorun core.clj: 3136 clojure.core/doall walk.clj: 47 clojure.walk/walk walk.clj: 65 clojure.walk/prewalk walk.clj: 126 clojure.walk/macroexpand-all walk.clj: 126 clojure.walk/macroexpand-all macroexpand.clj: 118 cider.nrepl.middleware.macroexpand/expand-clj/fn macroexpand.clj: 117 cider.nrepl.middleware.macroexpand/expand-clj macroexpand.clj: 111 cider.nrepl.middleware.macroexpand/expand-clj macroexpand.clj: 216 cider.nrepl.middleware.macroexpand/macroexpansion macroexpand.clj: 212 cider.nrepl.middleware.macroexpand/macroexpansion macroexpand.clj: 222 cider.nrepl.middleware.macroexpand/macroexpansion-reply macroexpand.clj: 221 cider.nrepl.middleware.macroexpand/macroexpansion-reply error_handling.clj: 160 cider.nrepl.middleware.util.error-handling/eval2258/fn MultiFn.java: 234 clojure.lang.MultiFn/invoke macroexpand.clj: 225 cider.nrepl.middleware.macroexpand/handle-macroexpand macroexpand.clj: 224 cider.nrepl.middleware.macroexpand/handle-macroexpand Var.java: 388 clojure.lang.Var/invoke nrepl.clj: 283 cider.nrepl/wrap-macroexpand/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 522 cider.nrepl/wrap-clojuredocs/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 428 cider.nrepl/wrap-stacktrace/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 204 cider.nrepl/wrap-info/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn interruptible_eval.clj: 154 nrepl.middleware.interruptible-eval/interruptible-eval/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 223 cider.nrepl/wrap-inspect/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn session.clj: 363 nrepl.middleware.session/add-stdin/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn load_file.clj: 81 nrepl.middleware.load-file/wrap-load-file/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 105 cider.nrepl/wrap-content-type/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn caught.clj: 97 nrepl.middleware.caught/wrap-caught/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 330 cider.nrepl/wrap-out/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 159 cider.nrepl/wrap-debug/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn print.clj: 234 nrepl.middleware.print/wrap-print/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 478 cider.nrepl/wrap-tracker/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn sideloader.clj: 104 nrepl.middleware.sideloader/wrap-sideloader/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn lookup.clj: 51 nrepl.middleware.lookup/wrap-lookup/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn session.clj: 309 nrepl.middleware.session/session/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 186 cider.nrepl/wrap-enlighten/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn server.clj: 130 nrepl.server/default-handler/fn server.clj: 22 nrepl.server/handle* server.clj: 19 nrepl.server/handle* server.clj: 39 nrepl.server/handle/fn core.clj: 2034 clojure.core/binding-conveyor-fn/fn AFn.java: 18 clojure.lang.AFn/call FutureTask.java: 264 java.util.concurrent.FutureTask/run ThreadPoolExecutor.java: 1128 java.util.concurrent.ThreadPoolExecutor/runWorker ThreadPoolExecutor.java: 628 java.util.concurrent.ThreadPoolExecutor$Worker/run Thread.java: 835 java.lang.Thread/run 1. Caused by clojure.lang.Compiler$CompilerException Error compiling NO_SOURCE_PATH at (0:0) #:clojure.error{:phase :macroexpansion, :line 0, :column 0, :source "NO_SOURCE_PATH", :symbol clojure.core/gen-interface} Compiler.java: 7023 clojure.lang.Compiler/macroexpand1 core.clj: 4012 clojure.core/macroexpand-1 core.clj: 4014 clojure.core/macroexpand walk.clj: 130 clojure.walk/macroexpand-all/fn walk.clj: 65 clojure.walk/prewalk walk.clj: 61 clojure.walk/prewalk core.clj: 2628 clojure.core/partial/fn core.clj: 2759 clojure.core/map/fn LazySeq.java: 42 clojure.lang.LazySeq/sval LazySeq.java: 51 clojure.lang.LazySeq/seq Cons.java: 39 clojure.lang.Cons/next RT.java: 713 clojure.lang.RT/next core.clj: 64 clojure.core/next core.clj: 3130 clojure.core/dorun core.clj: 3136 clojure.core/doall walk.clj: 47 clojure.walk/walk walk.clj: 65 clojure.walk/prewalk walk.clj: 126 clojure.walk/macroexpand-all walk.clj: 126 clojure.walk/macroexpand-all macroexpand.clj: 118 cider.nrepl.middleware.macroexpand/expand-clj/fn macroexpand.clj: 117 cider.nrepl.middleware.macroexpand/expand-clj macroexpand.clj: 111 cider.nrepl.middleware.macroexpand/expand-clj macroexpand.clj: 216 cider.nrepl.middleware.macroexpand/macroexpansion macroexpand.clj: 212 cider.nrepl.middleware.macroexpand/macroexpansion macroexpand.clj: 222 cider.nrepl.middleware.macroexpand/macroexpansion-reply macroexpand.clj: 221 cider.nrepl.middleware.macroexpand/macroexpansion-reply error_handling.clj: 160 cider.nrepl.middleware.util.error-handling/eval2258/fn MultiFn.java: 234 clojure.lang.MultiFn/invoke macroexpand.clj: 225 cider.nrepl.middleware.macroexpand/handle-macroexpand macroexpand.clj: 224 cider.nrepl.middleware.macroexpand/handle-macroexpand Var.java: 388 clojure.lang.Var/invoke nrepl.clj: 283 cider.nrepl/wrap-macroexpand/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 522 cider.nrepl/wrap-clojuredocs/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 428 cider.nrepl/wrap-stacktrace/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 204 cider.nrepl/wrap-info/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn interruptible_eval.clj: 154 nrepl.middleware.interruptible-eval/interruptible-eval/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 223 cider.nrepl/wrap-inspect/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn session.clj: 363 nrepl.middleware.session/add-stdin/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn load_file.clj: 81 nrepl.middleware.load-file/wrap-load-file/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 105 cider.nrepl/wrap-content-type/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn caught.clj: 97 nrepl.middleware.caught/wrap-caught/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 330 cider.nrepl/wrap-out/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 159 cider.nrepl/wrap-debug/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn print.clj: 234 nrepl.middleware.print/wrap-print/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 478 cider.nrepl/wrap-tracker/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn sideloader.clj: 104 nrepl.middleware.sideloader/wrap-sideloader/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn lookup.clj: 51 nrepl.middleware.lookup/wrap-lookup/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn session.clj: 309 nrepl.middleware.session/session/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn nrepl.clj: 186 cider.nrepl/wrap-enlighten/fn middleware.clj: 16 nrepl.middleware/wrap-conj-descriptor/fn server.clj: 130 nrepl.server/default-handler/fn server.clj: 22 nrepl.server/handle* server.clj: 19 nrepl.server/handle* server.clj: 39 nrepl.server/handle/fn core.clj: 2034 clojure.core/binding-conveyor-fn/fn AFn.java: 18 clojure.lang.AFn/call FutureTask.java: 264 java.util.concurrent.FutureTask/run ThreadPoolExecutor.java: 1128 java.util.concurrent.ThreadPoolExecutor/runWorker ThreadPoolExecutor.java: 628 java.util.concurrent.ThreadPoolExecutor$Worker/run Thread.java: 835 java.lang.Thread/run ```
vemv commented 3 years ago

Thanks!

Issue transfered to cider-nrepl which does the macroexpanding.

Might give this a shot soon enough.

vemv commented 3 years ago

I failed to reproduce the issue with this commit https://github.com/clojure-emacs/cider-nrepl/commit/1dfeef8535542e1a1c880555b8fcd699c77ffe8f so probably I will be out of ideas.

If you have the spare time, maybe you can try the same over a couple other JDKs; 12 is bit of an odd one. The LTS JDKs are 8, 11 and 17.

8 is the less likely one to give odd errors (such as the one you shared: Unhandled java.lang.ClassCastException class clojure.lang.Var$Unbound cannot be cast to class clojure.lang.DynamicClassLoader (clojure.lang.Var$Unbound and clojure.lang.DynamicClassLoader are in unnamed module of loader 'app'))

vemv commented 3 years ago

There's also a low but real chance that setting :jvm-opts ["-Dorchard.use-dynapath=false"] in deps.edn per https://github.com/clojure-emacs/orchard/tree/fa5bbb670bd502e59dfa61cfa8e65def38897239#configuration-options would leave the classloaders cleaner (unaltered), making the issue less likely.

jellelicht commented 3 years ago

@vemv thanks, I will be trying out your suggestions, and report back here. Were you able to at least reproduce the issue as I did (via cider)?

jellelicht commented 2 years ago

I can reproduce my issue using the linked repo with the following java versions, both with and without the use-dynapath settings.

I’ve not yet been able to test 17, as that has not been packaged for my distro yet. @vemv can I bother you to try to my reproducer in the repo I linked in my original issue?

I had one other person on the cider discord confirm my issue, but it would be nice if you can verify that you don’t see the issue at all (classic case of PEBKAC then on my end 😉).

vemv commented 2 years ago

I don't run a modern cider.el myself.

One thing you can try is renaming user.clj to something else - user has a special semantic and is loaded automatically / early.

jellelicht commented 2 years ago

I made sure to try with something that is not user.clj as well; same results. Would you be so kind as to move this back to the cider project in that case? TBH, I don’t really think the problem is here; since the macro-expanding the exact same code works when used in a lein project /w the same versions of pretty much everything else. Thanks for your help here either way!

EDIT: Since this issue only seems to occur for deps.edn projects, and not lein projects, I doubt we’ll be able to find + fix the problem here either way.

vemv commented 2 years ago

Would you be so kind as to move this back to the cider project in that case?

It's in bit of a grey zone, as we don't really know what's going on. We can default to avoid churn, I'm not sure you'd get any more help over the cider.el tracker.

n.b. as reported, so far this seems a rather low-impact corner case (which I'm willing to work through regardless).

One thing that would help is comparing the java processes. Try ps aux | grep java and please paste here the java processes that are ultimately running your code (e.g. the final JVM spawned by Lein, and the one JVM spawned by tools.deps).

It also would help to hit the macroexpand ns directly from a repl. This way we bypass a good chunk of the cider.el / nrepl machinery and can discard at least one hypothesis.

Cheers - V

vemv commented 2 years ago

btw an awesome repro might look like this:

clojure --eval "(require 'cider.nrepl.middleware.macroexpand), (cider.nrepl.middleware.macroexpand/,,, ,,,)"
jellelicht commented 2 years ago

Everything works as one would expect when directly calling into cider.nrepl.middleware.macroexpand, but I found a more fundamental issue (which might already give you some ideas on what the actual cause of the problem might be):

Given the following macro:

(defmacro dummy []                                                                                                                     
  (println  (deref clojure.lang.Compiler/LOADER))                                                                                      
  42)

And a deps.edn-based cider-repl will (correctly) M-x cider-macroexpand-all a call to (dummy) to 42. Kicker is, that the printen output will be:

#object[clojure.lang.Var$Unbound 0x3daadf30 Unbound: #<Var: --unnamed-->]

If one were to actually run (dummy) on the repl, one would see:

#object[clojure.lang.DynamicClassLoader 0x335cbf1b clojure.lang.DynamicClassLoader@335cbf1b]                                           
42

In a lein-based cider-repl, both will display a clojure.lang.DynamicCLassLoader! @vemv Does this help in any way?

vemv commented 2 years ago

Yes, sure! That's much useful.

I reckon that cider.nrepl.middleware.macroexpand could bind LOADER to a new DynamicClassLoader instance if it's currently unbound.

n.b., being unbound is its default state, so there's possibly more than one reason why it might remain unbound when invoking cider.nrepl.middleware.macroexpand. Which is a good thing here: we'd fix this issue, and possibly other unforeseen code paths.

Does this sound good to you?

vemv commented 2 years ago

(thanks for the 👍; I repro'd + fixed this in the meantime. PR soon)

jellelicht commented 2 years ago

Please Let me know if there’s something I can do to help test

jellelicht commented 2 years ago

@vemv I can confirm that it fixes my use-case! Thanks again!

plexus commented 2 years ago

object[clojure.lang.Var$Unbound 0x3daadf30 Unbound: #<Var: --unnamed-->]

This is misleading, this means the Var has no root binding, but it does usually have a thread-local binding during eval:

In a plain clj REPL:

user> [clojure.lang.Compiler/LOADER @clojure.lang.Compiler/LOADER]
;;=>
[#<Var: --unnamed--> #object[clojure.lang.DynamicClassLoader 0x7726e185 "clojure.lang.DynamicClassLoader@7726e185"]]
vemv commented 2 years ago

Got it, very interesting thanks!

Still, the open PR seems to make sense since its accompanying test would fail without the fix itself.

It can make sense if we consider during eval to not include the macroexpand phase?

i.e. the cider-macroexpand-all macroexpands, does not eval and therefore LOADER is truly unbound.

vemv commented 2 years ago

Released in CIDER 1.2 / cider-nrepl 0.27.4.