clojure-emacs / cider

The Clojure Interactive Development Environment that Rocks for Emacs
https://cider.mx
GNU General Public License v3.0
3.54k stars 645 forks source link

Error connecting to clojurescript shadow-cljs repl #2967

Closed evanlouie closed 3 years ago

evanlouie commented 3 years ago

This error started a couple days ago.

Expected behavior

When in a shadow-cljs project, I should be able to run:

Actual behavior

After running C-c C-x j m and selecting shadow an errors is printed:

error in process filter: parseclj-lex--leaf-token-value: Symbol’s function definition is void: :number
error in process filter: Symbol’s function definition is void: :number

When toggle-debug-on-error is enabled, the following is dumped:

Debugger entered--Lisp error: (void-function :number)
  :number(0)
  parseclj-lex--leaf-token-value(((:token-type . :keyword) (:form . ":source-paths") (:pos . 2)))
  (cons (parseclj-lex--leaf-token-value token) stack)
  (if (member (parseclj-lex-token-type token) (list :whitespace :comment)) stack (cons (parseclj-lex--leaf-token-value token) stack))
  parseedn-reduce-leaf((((:token-type . :lbrace) (:form . "{") (:pos . 1))) ((:token-type . :keyword) (:form . ":source-paths") (:pos . 2)) ((:tag-readers (shadow/env . identity))))
  funcall(parseedn-reduce-leaf (((:token-type . :lbrace) (:form . "{") (:pos . 1))) ((:token-type . :keyword) (:form . ":source-paths") (:pos . 2)) ((:tag-readers (shadow/env . identity))))
  (setq stack (funcall reduce-leaf stack token options))
  (cond ((parseclj-lex-leaf-token-p token) (setq stack (funcall reduce-leaf stack token options))) ((parseclj-lex-closing-token-p token) (setq stack (parseclj--reduce-coll stack token reduce-branch options))) (t (setq stack (cons token stack))))
  (while (not (or (and read-one (parseclj-single-value-p stack value-p)) (eq (parseclj-lex-token-type token) :eof))) (if (and fail-fast (parseclj-lex-error-p token)) (progn (parseclj--error "Invalid token at %s: %S" (a-get token :pos) (parseclj-lex-token-form token)))) (cond ((parseclj-lex-leaf-token-p token) (setq stack (funcall reduce-leaf stack token options))) ((parseclj-lex-closing-token-p token) (setq stack (parseclj--reduce-coll stack token reduce-branch options))) (t (setq stack (cons token stack)))) (let* ((top-value (parseclj--take-value stack value-p)) (opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)) new-stack) (while (and top-value opening-token) (setq new-stack (nthcdr (+ (length top-value) (length opening-token)) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value) options)) (setq top-value (parseclj--take-value stack value-p)) (setq opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)))) (let* ((top-value-1 (parseclj--take-value stack value-p)) (top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (opening-token (parseclj--take-token (nthcdr (+ (length top-value-1) (length top-value-2)) stack) value-p parseclj-lex--prefix-2-tokens)) new-stack) (while (and top-value-1 top-value-2 opening-token) (setq new-stack (nthcdr (apply #'+ (mapcar #'length (list top-value-1 top-value-2 opening-token))) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value-2 top-value-1) options)) (setq top-value-1 (parseclj--take-value stack value-p)) (setq top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (setq opening-token (parseclj--take-token (nthcdr (+ (length top-value-1) (length top-value-2)) stack) value-p parseclj-lex--prefix-2-tokens)))) (setq token (parseclj-lex-next)))
  (let ((fail-fast (a-get options :fail-fast t)) (read-one (a-get options :read-one)) (value-p (a-get options :value-p #'(lambda (e) (not (parseclj-lex-token-p e))))) (stack nil) (token (parseclj-lex-next))) (while (not (or (and read-one (parseclj-single-value-p stack value-p)) (eq (parseclj-lex-token-type token) :eof))) (if (and fail-fast (parseclj-lex-error-p token)) (progn (parseclj--error "Invalid token at %s: %S" (a-get token :pos) (parseclj-lex-token-form token)))) (cond ((parseclj-lex-leaf-token-p token) (setq stack (funcall reduce-leaf stack token options))) ((parseclj-lex-closing-token-p token) (setq stack (parseclj--reduce-coll stack token reduce-branch options))) (t (setq stack (cons token stack)))) (let* ((top-value (parseclj--take-value stack value-p)) (opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)) new-stack) (while (and top-value opening-token) (setq new-stack (nthcdr (+ (length top-value) (length opening-token)) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value) options)) (setq top-value (parseclj--take-value stack value-p)) (setq opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)))) (let* ((top-value-1 (parseclj--take-value stack value-p)) (top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (opening-token (parseclj--take-token (nthcdr (+ ... ...) stack) value-p parseclj-lex--prefix-2-tokens)) new-stack) (while (and top-value-1 top-value-2 opening-token) (setq new-stack (nthcdr (apply #'+ (mapcar ... ...)) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value-2 top-value-1) options)) (setq top-value-1 (parseclj--take-value stack value-p)) (setq top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (setq opening-token (parseclj--take-token (nthcdr (+ ... ...) stack) value-p parseclj-lex--prefix-2-tokens)))) (setq token (parseclj-lex-next))) (if fail-fast (progn (let* ((token (and t (seq-find ... stack)))) (if token (parseclj--error "At position %s, unmatched %S" (a-get token :pos) (parseclj-lex-token-type token)) nil)))) (if read-one (car (parseclj--take-value stack value-p)) (car (funcall reduce-branch nil (parseclj-lex-token :root "" 1) (reverse stack) options))))
  parseclj-parser(parseedn-reduce-leaf parseedn-reduce-branch ((:tag-readers (shadow/env . identity))))
  parseedn-read(((shadow/env . identity)))
  (car (parseedn-read '((shadow/env . identity))))
  (let ((hash (car (parseedn-read '((shadow/env . identity)))))) (cider--shadow-parse-builds hash))
  (progn (insert-file-contents shadow-edn) (let ((hash (car (parseedn-read '(...))))) (cider--shadow-parse-builds hash)))
  (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash (car (parseedn-read '...)))) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash (car (parseedn-read ...)))) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash (car ...))) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
  (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash ...)) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
  (if (file-exists-p shadow-edn) (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let (...) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))))
  (let ((shadow-edn (concat (clojure-project-dir) "shadow-cljs.edn"))) (if (file-exists-p shadow-edn) (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))))
  cider--shadow-get-builds()
  (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds))
  (or cider-shadow-default-options (car cider-shadow-watched-builds) (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds)))
  (cider-normalize-cljs-init-options (or cider-shadow-default-options (car cider-shadow-watched-builds) (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds))))
  (let* ((shadow-require "(require '[shadow.cljs.devtools.api :as shadow])") (default-build (cider-normalize-cljs-init-options (or cider-shadow-default-options (car cider-shadow-watched-builds) (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds))))) (watched-builds (or (mapcar #'cider-normalize-cljs-init-options cider-shadow-watched-builds) (list default-build))) (watched-builds-form (mapconcat #'(lambda (build) (format "(shadow/watch %s)" build)) watched-builds " ")) (user-build-form "(do %s %s (shadow/nrepl-select %s))") (default-build-form "(do %s (shadow/%s))")) (if (member default-build '(":browser-repl" ":node-repl")) (format default-build-form shadow-require (string-remove-prefix ":" default-build)) (format user-build-form shadow-require watched-builds-form default-build)))
  cider-shadow-cljs-init-form()
  funcall(cider-shadow-cljs-init-form)
  (if (symbolp repl-form) (funcall repl-form) repl-form)
  (if repl-form (if (symbolp repl-form) (funcall repl-form) repl-form) (user-error "No ClojureScript REPL type %s found.  Please make ..." repl-type))
  (let* ((repl-form (and t (car (cdr (seq-find #'... cider-cljs-repl-types)))))) (if repl-form (if (symbolp repl-form) (funcall repl-form) repl-form) (user-error "No ClojureScript REPL type %s found.  Please make ..." repl-type)))
  cider-cljs-repl-form(shadow)
  (let* ((cljs-type (plist-get params :cljs-repl-type)) (repl-init-form (cider-cljs-repl-form cljs-type))) (plist-put (plist-put params :repl-init-function #'(lambda nil (cider--check-cljs cljs-type) (set (make-local-variable 'cider-cljs-repl-type) cljs-type) (cider-nrepl-send-request (list "op" "eval" "ns" (cider-current-ns) "code" repl-init-form) (cider-repl-handler (current-buffer))) (if (and (buffer-live-p nrepl-server-buffer) cider-offer-to-open-cljs-app-in-browser) (progn (cider--offer-to-open-app-in-browser nrepl-server-buffer))))) :repl-init-form repl-init-form))
  (save-current-buffer (set-buffer (or (plist-get params :--context-buffer) (current-buffer))) (let* ((cljs-type (plist-get params :cljs-repl-type)) (repl-init-form (cider-cljs-repl-form cljs-type))) (plist-put (plist-put params :repl-init-function #'(lambda nil (cider--check-cljs cljs-type) (set (make-local-variable ...) cljs-type) (cider-nrepl-send-request (list "op" "eval" "ns" ... "code" repl-init-form) (cider-repl-handler ...)) (if (and ... cider-offer-to-open-cljs-app-in-browser) (progn ...)))) :repl-init-form repl-init-form)))
  cider--update-cljs-init-function((:project-dir "~/workspace/debugging-errror/" :host #("localhost" 0 9 (ivy-index 0 idx 0)) :port 53021 :cljs-repl-type shadow))
  (plist-put (cider--update-cljs-init-function (cider--update-cljs-type (cider--check-existing-session (cider--update-host-port (cider--update-project-dir params))))) :session-name nil)
  (plist-put (plist-put (cider--update-cljs-init-function (cider--update-cljs-type (cider--check-existing-session (cider--update-host-port (cider--update-project-dir params))))) :session-name nil) :repl-type 'pending-cljs)
  (cider-nrepl-connect (plist-put (plist-put (cider--update-cljs-init-function (cider--update-cljs-type (cider--check-existing-session (cider--update-host-port (cider--update-project-dir params))))) :session-name nil) :repl-type 'pending-cljs))
  cider-connect-cljs(nil)
  funcall-interactively(cider-connect-cljs nil)
  call-interactively(cider-connect-cljs nil nil)
  command-execute(cider-connect-cljs)

Steps to reproduce the problem

Project Setup:

Run it with either of:

Environment & Version information

Mac:

Linux:

Both running doom-emacs@develop

CIDER version information

C-c -C-x j m successfully starts a Clojure repl before erroring

;; CIDER 1.1.0snapshot, nREPL 0.8.3
;; Clojure 1.10.1, Java 11.0.9

Emacs version

GNU Emacs 27.1 (build 1, x86_64-apple-darwin20.2.0, Carbon Version 164 AppKit 2022.2) of 2021-01-14

Operating system

bbatsov commented 3 years ago

Seems that the EDN parser is struggling with something in your config. @plexus might have a better idea what exactly the problem is.

plexus commented 3 years ago

Quite bizarre, did you upgrade your emacs recently? it seems like cl-case is misbehaving

(defun parseclj-lex--leaf-token-value (token)
  "Parse the given leaf TOKEN to an Emacs Lisp value."
  (cl-case (parseclj-lex-token-type token)
    (:number (string-to-number (alist-get :form token)))
    (:nil nil)
    (:true t)
    (:false nil)
    (:symbol (intern (alist-get :form token)))
    (:keyword (intern (alist-get :form token)))
    (:string (parseclj-lex--string-value (alist-get :form token)))
    (:character (parseclj-lex--character-value (alist-get :form token)))))

It's treating that :number as a function call, rather than a value to match against.

evanlouie commented 3 years ago

@plexus

did you upgrade your emacs recently? it seems like cl-case is misbehaving

Sort of. I was using emacs-plus@27 for a while and everything was working fine. But then decided to give compiling gccemacs a go but that was just buggy in general. So I then reverted back to emacs-plus@27, and then this error started. I've since switched emacs-mac, but the error is still persisting.

I've also been able to recreate this issue on a newly formatted MacOS 11.1 machine with emacs-mac 27.1 running doom-emacs@develop.

evanlouie commented 3 years ago

Just recreated the the bug again on Ubuntu. Same setup, emacs@27.1 , doom-emacs@develop. For recreation, you can any of:

evanlouie commented 3 years ago

After some more digging I've found that this issue seems to be related to doom-emacs as it doesn't occur with spacemacs. I'll close this issue and open it on on the doom repo.

aiba commented 3 years ago

I just started hitting this same error. I'm using emacs-mac 27.2.

I think parseedn may be busted. When I evaluate (parseedn-read-str "{:a 1}") I get this stacktrace:

Debugger entered--Lisp error: (void-function :number)
  :number(0)
  parseclj-lex--leaf-token-value(((:token-type . :keyword) (:form . ":a") (:pos . 2)))
  parseedn-reduce-leaf((((:token-type . :lbrace) (:form . "{") (:pos . 1))) ((:token-type . :keyword) (:form . ":a") (:pos . 2)) ((:tag-readers)))
  parseclj-parser(parseedn-reduce-leaf parseedn-reduce-branch ((:tag-readers)))
  parseedn-read(nil)
  parseedn-read-str("{:a 1}")
  eval((parseedn-read-str "{:a 1}") nil)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)
vemv commented 3 years ago

Last batch of parseedn problems were fixed with https://github.com/clojure-emacs/cider/pull/3060 , are you running a recent cider.el snapshot?

If the problem persists, it would be helpful to know if it can also be observed with https://emacsformacosx.com/ which is much more vanilla than emacs-mac (of course trying it is just a one-off thing)

aiba commented 3 years ago

Same error with emacsformacosx (GNU Emacs 27.2 (build 1, x86_64-apple-darwin18.7.0, NS appkit-1671.60 Version 10.14.6 (Build 18G95)) of 2021-03-27).

I think I'm using the latest cider.el snapshot: CIDER 1.2.0snapshot (package: 20211003.947)

When you run (parseedn-read-str "{:a 1}"), you don't get that error?

vemv commented 3 years ago

When you run (parseedn-read-str "{:a 1}"), you don't get that error?

I can't help much here but perhaps something comes to mind to @plexus or @bbatsov ?

bbatsov commented 3 years ago

Probably deleting the installed packages CIDER, parseclj and parseedn and installing CIDER again is the best course of action at this point.

danjohansson commented 3 years ago

I'm having this problem as well. Downgrading parseedn and parseclj to 0.2.0 and the problem goes away. Upgrading and it comes back. Code to reproduce:

(require 'parseedn)
(parseedn-read-str "1")

It is reproducable in doom vanilla emacs sandbox too.

GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, cairo version 1.17.4) of 2021-03-26

plexus commented 3 years ago

As I've hinted at above this seems to be an Emacs regression in cl-case. The only fix we can do is rewrite the parseedn logic to not rely on cl-case.

plexus commented 3 years ago

I've opened PRs for parseclj and parseedn (parseedn relies on parseclj).

@evanlouie @aiba @danjohansson would be great if you could confirm that on your Emacs version, using parseclj/parseedn from git branch cl-case-to-cond, this works:

(require 'parseedn)
(parseedn-read-str "1")
;; => 1
danjohansson commented 3 years ago

@plexus I still have the same problem. The cl-case that gives the error in parseclj-lex--leaf-token-value has not changed in that branch?

danjohansson commented 3 years ago

If it helps: Running (parseclj-lex--leaf-token-value '((:token-type . :number) (:form . "1") (:pos . 1))) gives me the same error. But if I evaluate the function manually and run it again it works.

Can it be something with dynamic binding or evaluation order? @plexus

plexus commented 3 years ago

2021-10-12_101440_konsole

plexus commented 3 years ago

This is what parseclj-lex--leaf-token-value looks like now. If it still hase a cl-case then you are not on the right version.

(defun parseclj-lex--leaf-token-value (token)
  "Parse the given leaf TOKEN to an Emacs Lisp value."
  (let ((token-type (parseclj-lex-token-type token)))
    (cond
     ((eq :number token-type) (string-to-number (alist-get :form token)))
     ((eq :nil token-type) nil)
     ((eq :true token-type) t)
     ((eq :false token-type) nil)
     ((eq :symbol token-type) (intern (alist-get :form token)))
     ((eq :keyword token-type) (intern (alist-get :form token)))
     ((eq :string token-type) (parseclj-lex--string-value (alist-get :form token)))
     ((eq :character token-type) (parseclj-lex--character-value (alist-get :form token)))
     ((eq :symbolic-value token-type) (intern (substring (alist-get :form token) 2))))))
danjohansson commented 3 years ago

@plexus sorry, I only checked out parseedn. My bad.

plexus commented 3 years ago

No worries, I'll try to cut a release soon, but would be great if at least one person could confirm that this a) fixes the issue, and b) still seems to work in general :D

plexus commented 3 years ago

The split into two libraries is unfortunate, it should really be one code base, but it was one of the things we had to do to be considered for inclusion in MELPA.

danjohansson commented 3 years ago

With those fixes in place I get a different error: (I will double check my setup...)

Debugger entered--Lisp error: (void-function key)
  key((inst uuid))
  parseclj-alist-merge(((inst . #f(compiled-function (s) #<bytecode 0x156f1be68819>)) (uuid . #f(compiled-function (s) #<bytecode 0x156f1be6f629>))) nil)
  parseedn-reduce-branch(nil ((:token-type . :root) (:form . "") (:pos . 1)) (1) ((:tag-readers)))
  parseclj-parser(parseedn-reduce-leaf parseedn-reduce-branch ((:tag-readers)))
  parseedn-read(nil)
  parseedn-read-str("1")
  eval((parseedn-read-str "1") nil)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  eros-eval-last-sexp(nil)
  funcall-interactively(eros-eval-last-sexp nil)
  call-interactively(eros-eval-last-sexp nil nil)
  command-execute(eros-eval-last-sexp)
plexus commented 3 years ago

Does this work for you? Do a and b show up in the messages buffer?

(require 'seq)

(let ((keys '(a b)))
  (seq-doseq (key keys)
    (message "%S" key)))
danjohansson commented 3 years ago

Yes that works... :thinking:

plexus commented 3 years ago

Then I'm out of ideas... this is exactly what that code does. :/

https://github.com/clojure-emacs/parseclj/blob/master/parseclj-alist.el#L83-L89

plexus commented 3 years ago

I've replaced the seq-doseq with a mapcar, how about now?

danjohansson commented 3 years ago

Yeah the feeling I get is that some core functionality has been overriden by something. But I don't know elisp very well. I will see if I can get some time to dig a little deeper.

Thanks for the help so far! :+1:

danjohansson commented 3 years ago

@plexus I will give it a go

danjohansson commented 3 years ago

@plexus that did the trick! :)

plexus commented 3 years ago

Effing emacs... Emacs lisp is bad enough as it is, but stuff just randomly not working on some builds is absolutely infuriating. Anyway glad this works now, I'll release new parseclj/parseend versions.

plexus commented 3 years ago

Moral of the story, don't use any macros from cl-lib, seq, or map. It may seem convenient but it will come to bite you.

danjohansson commented 3 years ago

Thanks for the help! :+1:

aiba commented 3 years ago

Wow, thank you guys for tracking this down! Good to know about avoiding those elisp libs.

plexus commented 3 years ago

It's a real shame because these are the libs that make elisp bearable, but this is the second issue we have like this, first with cl-case and now with seq-doseq where part of a macro form is interpreted as a function call, but only for some people. It's really puzzling, and I'm not sure if it's an emacs regression/bug, a particular interplay with other packages, or if we're holding it wrong. Happy there are workarounds but it adds to the frustration of dealing with elisp.

plexus commented 3 years ago

Released in parseclj / parseedn v1.0.6

vemv commented 3 years ago

Should we bump the minimum requirement in cider.el?

plexus commented 3 years ago

yes, that would be a good idea.