Closed mjmeintjes closed 8 months ago
I think if you add an extra repo in your deps.edn
, clj-nix should be able to create the lock file, if the repository is public. But I didn't test that case.
But I'm not sure if datomic is really installed from a maven repo. I had that issue with Cognitec REBL, since you have to download it manually, and then run a bash script to install it on your local maven cache. That case isn't supported in clj-nix.
Not sure it's related, but I'm getting the same error..
When I add com.google.firebase/firebase-admin {:mvn/version "9.2.0"}
to the deps I get a Maven repo not found
when locking the dependencies, while building it with clj seems to work.
{:clojure.main/message
"Execution error (ExceptionInfo) at cljnix.utils/throw+ (utils.clj:23).\n\"Maven repo not found\" {:mvn-repos {\"central\" {:url \"https://repo1.maven.org/maven2/\"}, \"clojars\" {:url \"https://repo.clojars.org/\"}}, :file #object[sun.nio.fs.UnixPath 0x8cf7455 \"/var/folders/vk/n81gdx5n1pn6h_tkwq28n6mw0000gn/T/clj-cache7181250749841352763/mvn/io/grpc/grpc-bom/1.55.1/grpc-bom-1.55.1.pom\"]}\n\n",
:clojure.main/triage
{:clojure.error/class clojure.lang.ExceptionInfo,
:clojure.error/line 23,
:clojure.error/cause
"\"Maven repo not found\" {:mvn-repos {\"central\" {:url \"https://repo1.maven.org/maven2/\"}, \"clojars\" {:url \"https://repo.clojars.org/\"}}, :file #object[sun.nio.fs.UnixPath 0x8cf7455 \"/var/folders/vk/n81gdx5n1pn6h_tkwq28n6mw0000gn/T/clj-cache7181250749841352763/mvn/io/grpc/grpc-bom/1.55.1/grpc-bom-1.55.1.pom\"]}\n",
:clojure.error/symbol cljnix.utils/throw+,
:clojure.error/source "utils.clj",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type clojure.lang.ExceptionInfo,
:message
"\"Maven repo not found\" {:mvn-repos {\"central\" {:url \"https://repo1.maven.org/maven2/\"}, \"clojars\" {:url \"https://repo.clojars.org/\"}}, :file #object[sun.nio.fs.UnixPath 0x8cf7455 \"/var/folders/vk/n81gdx5n1pn6h_tkwq28n6mw0000gn/T/clj-cache7181250749841352763/mvn/io/grpc/grpc-bom/1.55.1/grpc-bom-1.55.1.pom\"]}\n",
:data
{:mvn-repos
{"central" {:url "https://repo1.maven.org/maven2/"},
"clojars" {:url "https://repo.clojars.org/"}},
:file
#object[sun.nio.fs.UnixPath 0x8cf7455 "/var/folders/vk/n81gdx5n1pn6h_tkwq28n6mw0000gn/T/clj-cache7181250749841352763/mvn/io/grpc/grpc-bom/1.55.1/grpc-bom-1.55.1.pom"]},
:at [cljnix.utils$throw_PLUS_ invokeStatic "utils.clj" 23]}],
:trace
[[cljnix.utils$throw_PLUS_ invokeStatic "utils.clj" 23]
[cljnix.utils$throw_PLUS_ invoke "utils.clj" 21]
[cljnix.utils$mvn_repo_info invokeStatic "utils.clj" 126]
[cljnix.utils$mvn_repo_info invoke "utils.clj" 113]
[cljnix.core$missing_mvn_deps$fn__21594 invoke "core.clj" 178]
[clojure.core$map$fn__5931$fn__5932 invoke "core.clj" 2759]
[clojure.core$filter$fn__5958$fn__5959 invoke "core.clj" 2823]
[clojure.core$map$fn__5931$fn__5932 invoke "core.clj" 2759]
[clojure.core$map$fn__5931$fn__5932 invoke "core.clj" 2759]
[clojure.core$filter$fn__5958$fn__5959 invoke "core.clj" 2823]
[clojure.lang.PersistentVector reduce "PersistentVector.java" 343]
[clojure.core$transduce invokeStatic "core.clj" 6946]
[clojure.core$into invokeStatic "core.clj" 6962]
[clojure.core$into invoke "core.clj" 6950]
[cljnix.core$missing_mvn_deps invokeStatic "core.clj" 170]
[cljnix.core$missing_mvn_deps invoke "core.clj" 162]
[cljnix.core$lock_file$fn__21666 invoke "core.clj" 368]
[clojure.core$completing$fn__8528 invoke "core.clj" 6930]
[clojure.core$map$fn__5931$fn__5932 invoke "core.clj" 2757]
[clojure.core$cat$fn__8851 invoke "core.clj" 7695]
[clojure.core$map$fn__5931$fn__5932 invoke "core.clj" 2757]
[clojure.core$map$fn__5931$fn__5932 invoke "core.clj" 2757]
[clojure.core$transduce invokeStatic "core.clj" 6948]
[clojure.core$transduce invoke "core.clj" 6933]
[cljnix.core$lock_file invokeStatic "core.clj" 352]
[cljnix.core$lock_file invoke "core.clj" 343]
[cljnix.core$_main invokeStatic "core.clj" 471]
[cljnix.core$_main doInvoke "core.clj" 466]
[clojure.lang.RestFn invoke "RestFn.java" 397]
[clojure.lang.AFn applyToHelper "AFn.java" 152]
[clojure.lang.RestFn applyTo "RestFn.java" 132]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.core$apply invokeStatic "core.clj" 667]
[clojure.main$main_opt invokeStatic "main.clj" 514]
[clojure.main$main_opt invoke "main.clj" 510]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]],
:cause
"\"Maven repo not found\" {:mvn-repos {\"central\" {:url \"https://repo1.maven.org/maven2/\"}, \"clojars\" {:url \"https://repo.clojars.org/\"}}, :file #object[sun.nio.fs.UnixPath 0x8cf7455 \"/var/folders/vk/n81gdx5n1pn6h_tkwq28n6mw0000gn/T/clj-cache7181250749841352763/mvn/io/grpc/grpc-bom/1.55.1/grpc-bom-1.55.1.pom\"]}\n",
:data
{:mvn-repos
{"central" {:url "https://repo1.maven.org/maven2/"},
"clojars" {:url "https://repo.clojars.org/"}},
:file
#object[sun.nio.fs.UnixPath 0x8cf7455 "/var/folders/vk/n81gdx5n1pn6h_tkwq28n6mw0000gn/T/clj-cache7181250749841352763/mvn/io/grpc/grpc-bom/1.55.1/grpc-bom-1.55.1.pom"]}}}
@das-monki could you paste the content of ~/.m2/repository/com/google/firebase/firebase-admin/9.2.0/_remote.repositories
?
If that file exists, could you try to delete ~/.m2/repository/com/google/firebase/firebase-admin/9.2.0
and generate the lock file again?
Thanks for the quick reply!
The content of the file:
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Nov 16 15:49:14 CET 2023
firebase-admin-9.2.0.jar>central=
firebase-admin-9.2.0.pom>central=
So the file ~/.m2/repository/io/grpc/grpc-bom/1.55.1/_remote.repositories
contains:
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Nov 16 16:11:33 CET 2023
grpc-bom-1.55.1.pom>central=
grpc-bom-1.55.1.pom>google-maven-central-copy=
But if I debug then the same file /tmp/my-deps/mvn/io/grpc/grpc-bom/1.55.1/_remote.repositories
in the temporary directory contains:
#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu Nov 16 19:08:10 CET 2023
grpc-bom-1.55.1.pom>google-maven-central-copy=
And since I don't have google-maven-central-copy
as a repo in my config it leads to the error from above.
I also tried adding it to deps.edn
with:
:mvn/repos {"google-maven-central-copy" {:url "https://maven-central.storage-download.googleapis.com/maven2"}}
but that doesn't help, because only the mvn/standard-repos
are considered when resolving. And anyway I guess something else goes wrong, because the file ~/.m2/repository/io/grpc/grpc-bom/1.55.1/_remote.repositories
contains a version that works.
I'm not that familiar with pom's and conventions around them, but for example /tmp/my-deps/mvn/com/google/cloud/google-cloud-pom-parent/1.13.0/google-cloud-pom-parent-1.13.0.pom
contains:
<repositories>
<repository>
<id>google-maven-central-copy</id>
<name>Google Maven Central copy</name>
<url>https://maven-central.storage-download.googleapis.com/maven2</url>
</repository>
<repository>
<id>maven-central</id>
<name>Maven Central</name>
<url>https://repo1.maven.org/maven2</url>
</repository>
</repositories>
And while tools-deps makes sure to first try central, then clojars, and then the rest, it identifies central as "central" and not "maven-central". Not sure if this is something that changed recently. For reference, the ordering of the repo's is in src/main/clojure/clojure/tools/deps/alpha/util/maven.clj
at remote-repos
.
It should be fixed in #103, you can try to generate a lock file with:
nix run github:jlesquembre/clj-nix/mvn-repos#deps-lock
@mjmeintjes What are the maven coordinates for datomic? Is it a private maven repo?
I'm not getting the Maven repo not found error anymore, but I am getting a StackOverflowError now:
> nix run github:jlesquembre/clj-nix/mvn-repos#deps-lock
Processing /home/matthys/work/projects/clj-tools/vendor/pack.alpha/deps.edn without aliases
Downloading: com/google/http-client/google-http-client-bom/1.32.1/google-http-client-bom-1.32.1.pom from central
Downloading: org/slf4j/slf4j-api/1.6.2/slf4j-api-1.6.2.pom from central
Downloading: org/clojure/tools.reader/0.10.0/tools.reader-0.10.0.pom from central
Downloading: org/apache/commons/commons-compress/1.8/commons-compress-1.8.pom from central
Downloading: org/clojure/pom.contrib/0.1.2/pom.contrib-0.1.2.pom from central
Downloading: org/apache/commons/commons-parent/33/commons-parent-33.pom from central
Downloading: org/apache/apache/13/apache-13.pom from central
Downloading: org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom from central
Processing /home/matthys/work/projects/clj-tools/vendor/clj-statecharts/deps.edn without aliases
Processing /home/matthys/work/projects/clj-tools/deps.edn without aliases
Execution error (StackOverflowError) at (REPL:1).
The StackOverFlowError stacktrace:
"Execution error (StackOverflowError) at (REPL:1).\nnull\n",
:clojure.main/triage
{:clojure.error/class java.lang.StackOverflowError,
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type java.lang.StackOverflowError,
:at
[clojure.lang.APersistentSet contains "APersistentSet.java" 34]}],
:trace
[[clojure.lang.APersistentSet contains "APersistentSet.java" 34]
[clojure.lang.PersistentHashSet cons "PersistentHashSet.java" 97]
[clojure.lang.PersistentHashSet cons "PersistentHashSet.java" 17]
[clojure.lang.RT conj "RT.java" 677]
[clojure.core$conj__5455 invokeStatic "core.clj" 87]
[clojure.core$distinct$step__6481$fn__6482$fn__6484
invoke
"core.clj"
5077]
[clojure.core$distinct$step__6481$fn__6482 invoke "core.clj" 5073]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]
[clojure.lang.RT seq "RT.java" 535]
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core$filter$fn__5962 invoke "core.clj" 2826]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
...
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core$concat$fn__5558 invoke "core.clj" 727]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]
[clojure.lang.RT seq "RT.java" 535]
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core$concat$fn__5558 invoke "core.clj" 727]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]
[clojure.lang.RT seq "RT.java" 535]
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core$concat$fn__5558 invoke "core.clj" 727]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]
[clojure.lang.RT seq "RT.java" 535]
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core$concat$fn__5558 invoke "core.clj" 727]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]
[clojure.lang.RT seq "RT.java" 535]
[clojure.core$seq__5467 invokeStatic "core.clj" 139]
[clojure.core$concat$fn__5558 invoke "core.clj" 727]
[clojure.lang.LazySeq sval "LazySeq.java" 42]
[clojure.lang.LazySeq seq "LazySeq.java" 51]]}}
@mjmeintjes could you share your deps.edn file? Or the maven coordinate causing problems?
The StackOverFlowError seems to be coming from utils/get-management-deps
but I think that is a different issue and will open a new issue for that.
For the Maven repo not found, the following deps.edn still throws the Maven repo not found
error.
{:paths ["src"]
:mvn/repos
{"jitpack" {:url "https://jitpack.io"}}
:deps
{com.github.kenglxn.qrgen/javase {:mvn/version "3.0.1"}}}
The following deps.edn throws an StackOverflowError:
{:paths ["src"]
:deps
{metosin/reitit {:mvn/version "0.5.15"}}}
@mjmeintjes thanks for the info. Most probably the StackOverflowError was introduced here: https://github.com/jlesquembre/clj-nix/pull/103/commits/f0bddacf59fb8ab278aabd7ff1c1e2573558adf2 I was trying to be clever and reduce the number of network requests.
I'll take a look to both issues next week.
@mjmeintjes I hope it's fixed now, could you try again?
nix run github:jlesquembre/clj-nix/mvn-repos#deps-lock
Seems to be getting there. No more StackOverflowErrors or Maven repo not found
, but now the following deps.edn throws an error when I run nix run github:jlesquembre/clj-nix/mvn-repos#deps-lock
:
{:paths ["src"]
:deps
{url-normalizer/url-normalizer {:mvn/version "0.5.3-1"}}}
1. Unhandled org.codehaus.plexus.util.xml.pull.XmlPullParserException
expected START_TAG or END_TAG not TEXT (position: TEXT seen
...</developers>\n `\n <p... @91:7)
MXParser.java: 1364 org.codehaus.plexus.util.xml.pull.MXParser/nextTag
MavenXpp3Reader.java: 2072 org.apache.maven.model.io.xpp3.MavenXpp3Reader/parseModel
MavenXpp3Reader.java: 3872 org.apache.maven.model.io.xpp3.MavenXpp3Reader/read
MavenXpp3Reader.java: 595 org.apache.maven.model.io.xpp3.MavenXpp3Reader/read
MavenXpp3Reader.java: 609 org.apache.maven.model.io.xpp3.MavenXpp3Reader/read
MavenXpp3Reader.java: 638 org.apache.maven.model.io.xpp3.MavenXpp3Reader/read
REPL: 42 cljnix.utils/pom
REPL: 37 cljnix.utils/pom
utils.clj: 327 cljnix.utils/get-parent
utils.clj: 325 cljnix.utils/get-parent
utils.clj: 347 cljnix.utils/get-parent-poms
utils.clj: 339 cljnix.utils/get-parent-poms
REPL: 58 cljnix.core/maven-deps/fn/fn
REPL: 58 cljnix.core/maven-deps/fn
core.clj: 2759 clojure.core/map/fn/fn
core.clj: 7682 clojure.core/preserving-reduced/fn
PersistentVector.java: 343 clojure.lang.PersistentVector/reduce
core.clj: 6885 clojure.core/reduce
core.clj: 7693 clojure.core/cat/fn
core.clj: 2759 clojure.core/map/fn/fn
core.clj: 2759 clojure.core/map/fn/fn
core.clj: 2823 clojure.core/filter/fn/fn
protocols.clj: 49 clojure.core.protocols/iter-reduce
protocols.clj: 75 clojure.core.protocols/fn
protocols.clj: 75 clojure.core.protocols/fn
protocols.clj: 13 clojure.core.protocols/fn/G
core.clj: 6947 clojure.core/transduce
core.clj: 6962 clojure.core/into
core.clj: 6950 clojure.core/into
REPL: 32 cljnix.core/maven-deps
REPL: 30 cljnix.core/maven-deps
core.clj: 236 cljnix.core/get-deps!
core.clj: 216 cljnix.core/get-deps!
core.clj: 355 cljnix.core/lock-file/fn
core.clj: 2759 clojure.core/map/fn/fn
core.clj: 7682 clojure.core/preserving-reduced/fn
PersistentVector.java: 343 clojure.lang.PersistentVector/reduce
core.clj: 6885 clojure.core/reduce
core.clj: 7693 clojure.core/cat/fn
core.clj: 2759 clojure.core/map/fn/fn
core.clj: 2759 clojure.core/map/fn/fn
ArrayChunk.java: 58 clojure.lang.ArrayChunk/reduce
protocols.clj: 136 clojure.core.protocols/fn
protocols.clj: 124 clojure.core.protocols/fn
protocols.clj: 19 clojure.core.protocols/fn/G
protocols.clj: 31 clojure.core.protocols/seq-reduce
protocols.clj: 75 clojure.core.protocols/fn
protocols.clj: 75 clojure.core.protocols/fn
protocols.clj: 13 clojure.core.protocols/fn/G
core.clj: 6947 clojure.core/transduce
core.clj: 6933 clojure.core/transduce
core.clj: 351 cljnix.core/lock-file
core.clj: 342 cljnix.core/lock-file
REPL: 526 cljnix.core/eval55058
I've traced the XmlPullParserException to line 90 of this POM. There seems to be a ` character that shouldn't be there, making it invalid XML.
@jlesquembre Awesome! Just tried, and locking firebase now works! 🥳 Thanks a lot!! 😊
@mjmeintjes Thanks for you reports, the invalid pom.xml case should be fixed now on #103 . I didn't expect maven to accept invalid POMs on maven central.
@jlesquembre Awesome! Just tried, and locking firebase now works! 🥳 Thanks a lot!! 😊
@das-monki awesome, thanks for testing!
Thanks for all the work.
I'm getting the following error when running nix run github:jlesquembre/clj-nix/mvn-repos#deps-lock -- --deps-include ./deps.edn
in my repo.
{:clojure.main/message
"Execution error (DirectoryNotEmptyException) at sun.nio.fs.UnixFileSystemProvider/implDelete (UnixFileSystemProvider.java:246).\n/tmp/clj-cache17376787517332311277/mvn\n",
:clojure.main/triage
{:clojure.error/class java.nio.file.DirectoryNotEmptyException,
:clojure.error/line 246,
:clojure.error/cause "/tmp/clj-cache17376787517332311277/mvn",
:clojure.error/symbol sun.nio.fs.UnixFileSystemProvider/implDelete,
:clojure.error/source "UnixFileSystemProvider.java",
:clojure.error/phase :execution},
:clojure.main/trace
{:via
[{:type java.nio.file.DirectoryNotEmptyException,
:message "/tmp/clj-cache17376787517332311277/mvn",
:at
[sun.nio.fs.UnixFileSystemProvider
implDelete
"UnixFileSystemProvider.java"
246]}],
:trace
[[sun.nio.fs.UnixFileSystemProvider
implDelete
"UnixFileSystemProvider.java"
246]
[sun.nio.fs.AbstractFileSystemProvider
delete
"AbstractFileSystemProvider.java"
105]
[java.nio.file.Files delete "Files.java" 1152]
[babashka.fs$delete invokeStatic "fs.cljc" 545]
[babashka.fs$delete invoke "fs.cljc" 539]
[babashka.fs$delete_tree$fn__4004 invoke "fs.cljc" 567]
[babashka.fs$walk_file_tree$reify__3927
postVisitDirectory
"fs.cljc"
212]
[java.nio.file.Files walkFileTree "Files.java" 2827]
[babashka.fs$walk_file_tree invokeStatic "fs.cljc" 204]
[babashka.fs$walk_file_tree invoke "fs.cljc" 185]
[babashka.fs$delete_tree invokeStatic "fs.cljc" 562]
[babashka.fs$delete_tree invoke "fs.cljc" 558]
[cljnix.core$lock_file invokeStatic "core.clj" 350]
[cljnix.core$lock_file invoke "core.clj" 342]
[cljnix.core$_main invokeStatic "core.clj" 473]
[cljnix.core$_main doInvoke "core.clj" 468]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.core$apply invokeStatic "core.clj" 667]
[clojure.main$main_opt invokeStatic "main.clj" 514]
[clojure.main$main_opt invoke "main.clj" 510]
[clojure.main$main invokeStatic "main.clj" 664]
[clojure.main$main doInvoke "main.clj" 616]
[clojure.lang.RestFn applyTo "RestFn.java" 137]
[clojure.lang.Var applyTo "Var.java" 705]
[clojure.main main "main.java" 40]],
:cause "/tmp/clj-cache17376787517332311277/mvn"}}
I've tried to manual delete the tmp directory, but the error happens each time I run the command.
The contents of that tmp folder are:
/tmp/clj-cache17376787517332311277/mvn/:
bardo chlorine tube-alloys
/tmp/clj-cache17376787517332311277/mvn/bardo/:
bardo
/tmp/clj-cache17376787517332311277/mvn/bardo/bardo/:
0.1.2-SNAPSHOT
/tmp/clj-cache17376787517332311277/mvn/bardo/bardo/0.1.2-SNAPSHOT/:
resolver-status.properties
/tmp/clj-cache17376787517332311277/mvn/chlorine/:
js
/tmp/clj-cache17376787517332311277/mvn/chlorine/js/:
1.6.4-SNAPSHOT
/tmp/clj-cache17376787517332311277/mvn/chlorine/js/1.6.4-SNAPSHOT/:
resolver-status.properties
/tmp/clj-cache17376787517332311277/mvn/tube-alloys/:
redef-methods
/tmp/clj-cache17376787517332311277/mvn/tube-alloys/redef-methods/:
1.0.0-SNAPSHOT
/tmp/clj-cache17376787517332311277/mvn/tube-alloys/redef-methods/1.0.0-SNAPSHOT/:
resolver-status.properties
@mjmeintjes looks like it fails because the tmp directory can't be deleted. But it doesn't make too much sense if you can delete the directory with a rm -r ...
@mjmeintjes I merged #103 , but let's try to fix your latest error, can you try the delete-error
branch?
nix run github:jlesquembre/clj-nix/delete-error#deps-lock
Running nix run github:jlesquembre/clj-nix/delete-error#deps-lock
successfully completes now, thanks. I haven't tested using the lockfile yet, but I think we can close this issue.
Does clj-nix support other maven repos?
I am trying to create the lockfile, but I get the following error. I'm assuming it is something to do with datomic using a different maven repo.