Neil completely crashes on fresh install

[keturiosakys]

Hey! Just trying out neil as I'm wrapping my head around Clojure tools.

I've installed it with brew on macOS 13.4.1. Immediately running neil crashes it with the following error:

[borkdude]

I'll take a look when I have time, currently traveling. Would appreciate others taking a look.

[teodorlu]

@keturiosakys hey!

Do you have any other files in the folder you're running neil? Specifically, do you have a bb.edn file?

[keturiosakys]

@teodorlu hey, thanks for your reply! Nope, there's a deps.edn file as I'm using that for dependencies but no bb.edn

[teodorlu]

Ok, cool! Let's try to fix this.

• Can you share the content of your deps.edn file?
• Do you get the same error if you run Neil in a different folder?

I'm asking because there might be errors in the deps.edn file. Also, perhaps I'll be able to reproduce your error. If I can reproduce, it'll be much easier for me to find a fix.

[keturiosakys]

The issue appears no matter which directory I run neil in.

Here's the deps.edn file contents though:

{:paths ["src" "resources"]
 :deps {org.clojure/clojure {:mvn/version "1.11.1"}
        io.github.nextjournal/clerk {:mvn/version "0.14.919"}
        clj-http/clj-http {:mvn/version "3.12.3"}}
 :aliases
 {:run-m {:main-opts ["-m" "funda-analyzer.funda-analyzer"]}
  :run-x {:ns-default funda-analyzer.funda-analyzer
          :exec-fn greet
          :exec-args {:name "Clojure"}}}

 :build {:deps {io.github.seancorfield/build-clj
                {:git/tag "v0.6.3" :git/sha "9b8e09b"
                 :deps/root "slim"
                 :ns-default build}}
         :test {:extra-paths ["test"]
                :extra-deps {org.clojure/test.check {:mvn/version "1.1.1"}
                             io.github.cognitect-labs/test-runner
                             {:git/tag "v0.5.1" :git/sha "dfb30dd"}}}}}

[borkdude]

What output do you get for bb —version?

It could be that there is a pretty old bb on your path?

On Wed, 5 Jul 2023 at 11:06, Laurynas Keturakis @.***> wrote:

The issue appears no matter which directory I run neil in.

Here's the deps.edn file contents though:

{:paths ["src" "resources"] :deps {org.clojure/clojure {:mvn/version "1.11.1"} io.github.nextjournal/clerk {:mvn/version "0.14.919"} clj-http/clj-http {:mvn/version "3.12.3"}} :aliases {:run-m {:main-opts ["-m" "funda-analyzer.funda-analyzer"]} :run-x {:ns-default funda-analyzer.funda-analyzer :exec-fn greet :exec-args {:name "Clojure"}}}

:build {:deps {io.github.seancorfield/build-clj {:git/tag "v0.6.3" :git/sha "9b8e09b" :deps/root "slim" :ns-default build}} :test {:extra-paths ["test"] :extra-deps {org.clojure/test.check {:mvn/version "1.1.1"} io.github.cognitect-labs/test-runner {:git/tag "v0.5.1" :git/sha "dfb30dd"}}}}}

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621339622, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBQYQM2NC546P7NUUCTXOUVAPANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[keturiosakys]

bb --version outputs babashka v0.8.2

I'm not sure if that's the latest as I'm seeing v1.* versions on Github but installing on brew with borkdude/brew/babashka that's the latest that I get

[borkdude]

Can you find out if this bb binary came with home brew or if it’s something you installed yourself? It’s very old and this is very likely the problem

On Wed, 5 Jul 2023 at 11:45, Laurynas Keturakis @.***> wrote:

bb --version outputs babashka v0.8.2

I'm not sure if that's the latest as I'm seeing v1.* versions on Github but installing on brew with borkdude/brew/babashka that's the latest that I get

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621411661, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBR65CEPH2I76EHIL2TXOUZUDANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[keturiosakys]

Indeed that was an issue (apparently I had two bb installed that conflicted) - my bad!

I am running into another crash now, however. bb on its own works fine, but neil crashes.

I'm guessing I ended up in some sort of weird state between the installs. Will try a completely clean reinstall of bb and neil

See the stack trace below:

~/C/k/funda-analyzer main• ❱ neil
Error: Could not find or load main class clojure.main
Caused by: java.lang.ClassNotFoundException: clojure.main
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:
Data:     {:proc #object[java.lang.ProcessImpl 0x42dc82ba "Process[pid=45903, exitValue=1]"], :exit 1, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x3e6e8737 "java.lang.ProcessBuilder$NullOutputStream@3e6e8737"], :out #object[java.lang.ProcessImpl$ProcessPipeInputStream 0x4b611413 "java.lang.ProcessImpl$ProcessPipeInputStream@4b611413"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x5e8022c2 "java.lang.ProcessBuilder$NullInputStream@5e8022c2"], :prev nil, :cmd ["/Users/laurynas-fp/.asdf/shims/java" "-XX:-OmitStackTraceInFastThrow" "-classpath" "/Users/laurynas-fp/.deps.clj/1.11.1.1347/ClojureTools/clojure-tools-1.11.1.1347.jar" "clojure.main" "-m" "clojure.tools.deps.script.make-classpath2" "--config-user" "" "--config-project" "__babashka_no_deps_file__.edn" "--basis-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.basis" "--cp-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.cp" "--jvm-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.jvm" "--main-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.main" "--manifest-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.manifest" "--config-data" "{:deps {borkdude/rewrite-edn {:mvn/version \"0.4.6\"}, org.babashka/spec.alpha {:git/url \"https://github.com/babashka/spec.alpha\", :git/sha \"1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78\"}, io.github.seancorfield/deps-new {:git/url \"https://github.com/seancorfield/deps-new\", :git/tag \"v0.5.0\", :git/sha \"48bf01e\"}, org.babashka/cli {:mvn/version \"0.6.45\"}, version-clj/version-clj {:mvn/version \"2.0.2\"}, org.babashka/http-client {:mvn/version \"0.1.4\"}}, :aliases {:org.babashka/defaults {:replace-paths [], :classpath-overrides {org.clojure/clojure \"\", org.clojure/spec.alpha \"\", org.clojure/core.specs.alpha \"\"}}}}" "-A::org.babashka/defaults"], :type :babashka.process/error}
Location: /opt/homebrew/bin/neil:8:1

----- Context ------------------------------------------------------------------
 4: ;; DO NOT EDIT DIRECTLY. GENERATED FROM scr/babashka/neil.clj
 5:
 6: (require '[babashka.deps :as deps])
 7:
 8: (deps/add-deps '{:deps {borkdude/rewrite-edn {:mvn/version "0.4.6"}
    ^---
 9:                         org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha"
10:                                                  :git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"}
11:                         io.github.seancorfield/deps-new {:git/url "https://github.com/seancorfield/deps-new"
12:                                                          :git/tag "v0.5.0"
13:                                                          :git/sha "48bf01e"}

----- Stack trace --------------------------------------------------------------
babashka.process/check                          - <built-in>
babashka.process/shell                          - <built-in>
babashka.impl.deps/add-deps/fn--27048           - <built-in>
borkdude.deps/-main                             - <built-in>
clojure.core/apply                              - <built-in>
... (run with --debug to see elided elements)
clojure.core/apply                              - <built-in>
clojure.core/with-bindings*                     - <built-in>
babashka.impl.deps/add-deps/fn--27055           - <built-in>
babashka.impl.deps/add-deps                     - <built-in>
user                                            - /opt/homebrew/bin/neil:8:1

[borkdude]

Can you make a backup of $HOME/.deps.clj and attach it as a zip file. Then delete this directory and try again.

On Wed, 5 Jul 2023 at 12:28, Laurynas Keturakis @.***> wrote:

Indeed that was an issue (apparently I had two bb installed that conflicted) - my bad!

I am running into another crash now, however. bb on its own works fine, but neil crashes.

See the stack trace below:

~/C/k/funda-analyzer main• ❱ neil Error: Could not find or load main class clojure.main Caused by: java.lang.ClassNotFoundException: clojure.main ----- Error -------------------------------------------------------------------- Type: clojure.lang.ExceptionInfo Message: Data: {:proc #object[java.lang.ProcessImpl 0x42dc82ba "Process[pid=45903, exitValue=1]"], :exit 1, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x3e6e8737 @."], :out #object[java.lang.ProcessImpl$ProcessPipeInputStream 0x4b611413 @."], :err #object[java.lang.ProcessBuilder$NullInputStream 0x5e8022c2 @.***"], :prev nil, :cmd ["/Users/laurynas-fp/.asdf/shims/java" "-XX:-OmitStackTraceInFastThrow" "-classpath" "/Users/laurynas-fp/.deps.clj/1.11.1.1347/ClojureTools/clojure-tools-1.11.1.1347.jar" "clojure.main" "-m" "clojure.tools.deps.script.make-classpath2" "--config-user" "" "--config-project" "__babashka_no_deps_file__.edn" "--basis-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.basis" "--cp-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.cp" "--jvm-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.jvm" "--main-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.main" "--manifest-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.manifest" "--config-data" "{:deps {borkdude/rewrite-edn {:mvn/version \"0.4.6\"}, org.babashka/spec.alpha {:git/url \"https://github.com/babashka/spec.alpha\", :git/sha \"1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78\"}, io.github.seancorfield/deps-new {:git/url \"https://github.com/seancorfield/deps-new\", :git/tag \"v0.5.0\", :git/sha \"48bf01e\"}, org.babashka/cli {:mvn/version \"0.6.45\"}, version-clj/version-clj {:mvn/version \"2.0.2\"}, org.babashka/http-client {:mvn/version \"0.1.4\"}}, :aliases {:org.babashka/defaults {:replace-paths [], :classpath-overrides {org.clojure/clojure \"\", org.clojure/spec.alpha \"\", org.clojure/core.specs.alpha \"\"}}}}" "-A::org.babashka/defaults"], :type :babashka.process/error} Location: /opt/homebrew/bin/neil:8:1

----- Context ------------------------------------------------------------------ 4: ;; DO NOT EDIT DIRECTLY. GENERATED FROM scr/babashka/neil.clj 5: 6: (require '[babashka.deps :as deps]) 7: 8: (deps/add-deps '{:deps {borkdude/rewrite-edn {:mvn/version "0.4.6"} ^--- 9: org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha" 10: :git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"} 11: io.github.seancorfield/deps-new {:git/url "https://github.com/seancorfield/deps-new" 12: :git/tag "v0.5.0" 13: :git/sha "48bf01e"}

----- Stack trace -------------------------------------------------------------- babashka.process/check - babashka.process/shell - babashka.impl.deps/add-deps/fn--27048 - borkdude.deps/-main - clojure.core/apply - ... (run with --debug to see elided elements) clojure.core/apply - clojure.core/with-bindings* - babashka.impl.deps/add-deps/fn--27055 - babashka.impl.deps/add-deps - user - /opt/homebrew/bin/neil:8:1

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621481064, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBT6LEJ2JXDW5AN6KYTXOU6VLANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[borkdude]

Also delete $HOME/.clojure/.cpcache

On Wed, 5 Jul 2023 at 12:47, Michiel Borkent @.***> wrote:

Can you make a backup of $HOME/.deps.clj and attach it as a zip file. Then delete this directory and try again.

On Wed, 5 Jul 2023 at 12:28, Laurynas Keturakis @.***> wrote:

Indeed that was an issue (apparently I had two bb installed that conflicted) - my bad!

I am running into another crash now, however. bb on its own works fine, but neil crashes.

See the stack trace below:

~/C/k/funda-analyzer main• ❱ neil Error: Could not find or load main class clojure.main Caused by: java.lang.ClassNotFoundException: clojure.main ----- Error -------------------------------------------------------------------- Type: clojure.lang.ExceptionInfo Message: Data: {:proc #object[java.lang.ProcessImpl 0x42dc82ba "Process[pid=45903, exitValue=1]"], :exit 1, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x3e6e8737 @."], :out #object[java.lang.ProcessImpl$ProcessPipeInputStream 0x4b611413 @."], :err #object[java.lang.ProcessBuilder$NullInputStream 0x5e8022c2 @.***"], :prev nil, :cmd ["/Users/laurynas-fp/.asdf/shims/java" "-XX:-OmitStackTraceInFastThrow" "-classpath" "/Users/laurynas-fp/.deps.clj/1.11.1.1347/ClojureTools/clojure-tools-1.11.1.1347.jar" "clojure.main" "-m" "clojure.tools.deps.script.make-classpath2" "--config-user" "" "--config-project" "__babashka_no_deps_file__.edn" "--basis-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.basis" "--cp-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.cp" "--jvm-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.jvm" "--main-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.main" "--manifest-file" "/Users/laurynas-fp/.clojure/.cpcache/158BEBBC72DF2CC48E598157BC4F6490.manifest" "--config-data" "{:deps {borkdude/rewrite-edn {:mvn/version \"0.4.6\"}, org.babashka/spec.alpha {:git/url \"https://github.com/babashka/spec.alpha\", :git/sha \"1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78\"}, io.github.seancorfield/deps-new {:git/url \"https://github.com/seancorfield/deps-new\", :git/tag \"v0.5.0\", :git/sha \"48bf01e\"}, org.babashka/cli {:mvn/version \"0.6.45\"}, version-clj/version-clj {:mvn/version \"2.0.2\"}, org.babashka/http-client {:mvn/version \"0.1.4\"}}, :aliases {:org.babashka/defaults {:replace-paths [], :classpath-overrides {org.clojure/clojure \"\", org.clojure/spec.alpha \"\", org.clojure/core.specs.alpha \"\"}}}}" "-A::org.babashka/defaults"], :type :babashka.process/error} Location: /opt/homebrew/bin/neil:8:1

----- Context ------------------------------------------------------------------ 4: ;; DO NOT EDIT DIRECTLY. GENERATED FROM scr/babashka/neil.clj 5: 6: (require '[babashka.deps :as deps]) 7: 8: (deps/add-deps '{:deps {borkdude/rewrite-edn {:mvn/version "0.4.6"} ^--- 9: org.babashka/spec.alpha {:git/url "https://github.com/babashka/spec.alpha" 10: :git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"} 11: io.github.seancorfield/deps-new {:git/url "https://github.com/seancorfield/deps-new" 12: :git/tag "v0.5.0" 13: :git/sha "48bf01e"}

----- Stack trace -------------------------------------------------------------- babashka.process/check - babashka.process/shell - babashka.impl.deps/add-deps/fn--27048 - borkdude.deps/-main - clojure.core/apply - ... (run with --debug to see elided elements) clojure.core/apply - clojure.core/with-bindings* - babashka.impl.deps/add-deps/fn--27055 - babashka.impl.deps/add-deps - user - /opt/homebrew/bin/neil:8:1

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621481064, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBT6LEJ2JXDW5AN6KYTXOU6VLANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[keturiosakys]

No dice :( I removed/reinstalled both and tried with deleting .clojure/.cpcache as well

[borkdude]

Have you attached the zip file so I can reproduce the issue?

On Wed, 5 Jul 2023 at 15:05, Laurynas Keturakis @.***> wrote:

No dice :( I removed/reinstalled both and tried with deleting .clojure/.cpcache as well

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621720875, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBRVO6EUB7UPYM3ACPTXOVRCDANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[keturiosakys]

oh apologies, missed the instructions regarding .deps.clj. Deleting that and rerunning neil has worked!

I diff'ed my corrupt .deps.clj I backed up and the new one and it seems that the former was missing example-deps.edn and tools.edn files that I imagine caused the errors. The corrupt version also had an unexpanded but corrupt tools.zip archive.

Not exactly sure why the installation got corrupted, but the issue is now resolved.

Thanks a lot for your help!

[borkdude]

Can you please upload a zip of your corrrupt .deps.clj directory as twice requested? Only then will I be able to fix this problem in the future.

On Wed, 5 Jul 2023 at 15:48, Laurynas Keturakis @.***> wrote:

oh apologies, missed the instructions regarding .deps.clj. Deleting that and rerunning neil has worked!

I diff'ed my corrupt .deps.clj I backed up and the new one and it seems that the former was missing example-deps.edn and tools.edn files that I imagine caused the errors. The corrupt version also had an unexpanded but corrupt tools.zip archive.

Not exactly sure why the installation got corrupted, but the issue is now resolved.

Thanks a lot for your help!

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621799366, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBTJXPT7E5N4YSCFG4LXOVWDXANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[keturiosakys]

Sure! The zipped file was too big for GitHub (25mb limit) so here's a link for download: https://we.tl/t-Omzy3FDYii

[borkdude]

Thank you! I’ll take a look soon.

On Wed, 5 Jul 2023 at 16:57, Laurynas Keturakis @.***> wrote:

Sure! The zipped file was too big for GitHub (25mb limit) so here's a link for download: https://we.tl/t-Omzy3FDYii

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1621937454, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBXEIII77NKG5J642PLXOV6FRANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[borkdude]

@keturiosakys Could it be that upon the first time that bb tried to download deps.clj's tools jar, you aborted the operation midway?

You should also have seen something like:

CRC check failed when unzipping zip-file /Users/borkdude/.deps.clj/1.11.1.1347/ClojureTools/tools.zip, entry: ClojureTools/exec.jar
The tools zip file may have not been succesfully downloaded.nPlease report this problem and keep a backup of the tools zip file as a repro.nYou can try again by removing the $HOME/.deps.clj folder.
C

in babashka v1.3.181. What version are you using right now?

[keturiosakys]

I'm on the same version of babashka currently babashka v1.3.181.

And yep I think I can confirm it was an interrupted operation. I queried my shell history for commands that ended with 130 (interrupted) exit code and it appears I ran neil bb, which was the first run of neil after re-downloading, and then for some reason cancelled the run immediately.

[borkdude]

Thanks for the feedback. I'll think more about how to make the installation procedure more robust.

On Wed, Jul 5, 2023 at 11:26 PM Laurynas Keturakis @.***> wrote:

I'm on the same version of babashka currently babashka v1.3.181.

And yep I think I can confirm it was an interrupted install. I queried my shell history for commands that ended with 130 (interrupted) exit code and it appears I ran neil bb and for some reason cancelled the run immediately. [image: CleanShot 2023-07-05 at 23 21 @.*** https://user-images.githubusercontent.com/17225208/251286722-75216fb6-12c0-45d6-8b99-6ac7e3f0ed0c.png

— Reply to this email directly, view it on GitHub https://github.com/babashka/neil/issues/184#issuecomment-1622539169, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACFSBSLGOGKXALM2BM73PLXOXLXXANCNFSM6AAAAAAZ6E6SK4 . You are receiving this because you commented.Message ID: @.***>

-- https://www.michielborkent.nl https://www.eetvoorjeleven.nu

[sg-qwt]

Also today I found neil from Nixpkgs is broken as well. https://github.com/NixOS/nixpkgs/issues/258132

I don't know why neil is trying to fetch clojure tools from internet. Is there ways to disable the downloading behavior and add dependencies statically?

[borkdude]

We can make neil more robust by not fetching dependencies at runtime, this means we'll have to distribute it as an uberjar.

Currently some neil dependencies are fetched on the first run: https://github.com/babashka/neil/blob/d76462a5405b32844fd970bc9366fd23975cdddb/neil#L8-L16

[sg-qwt]

We can make neil more robust by not fetching dependencies at runtime, this means we'll have to distribute it as an uberjar.

Currently some neil dependencies are fetched on the first run:

https://github.com/babashka/neil/blob/d76462a5405b32844fd970bc9366fd23975cdddb/neil#L8-L16

The uberjar approach without runtime fetching seems reasonable to me.

The issue I reported has actually nothing to do with this, it's nix specific. It's caused by nix's particular way of wrapped babashka that case. https://github.com/NixOS/nixpkgs/pull/257473

[borkdude]

In the meanwhile deps.clj has been made more robust.