search

replikativ / datahike

A fast, immutable, distributed & compositional Datalog engine for everyone.
https://datahike.io
Eclipse Public License 1.0
1.62k stars 95 forks source link

[Bug]: unneeded dependencies pulled #634

Closed rome-user closed 11 months ago

rome-user commented 1 year ago

What version of Datahike are you using?

latest

What version of Java are you using?

17

What operating system are you using?

N/A

What database EDN configuration are you using?

N/A

Describe the bug

When project was using Leiningen, ClojureScript and some other libraries were declared with :scope "provided". This allowed users to supply e.g. their own ClojureScript dependency, or simply not provide it if they did not use the functionality requiring ClojureScript.

What is the expected behaviour?

I can use this in pure Clojure project without needing to pull in ClojureScript, cljs-node-io, etc.

How can the behaviour be reproduced?

Running the following commands

lein new example
cd example
lein change :dependencies conj '[io.replikativ/datahike "0.6.1544"]'
lein deps :tree

we see various CLJS dependencies pulled in, despite not using CLJS

Possibly confusing dependencies found:
[io.replikativ/datahike "0.6.1544"] -> [com.taoensso/timbre "5.2.1"]
 overrides
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/konserve "0.7.311"] -> [com.taoensso/timbre "6.0.1"]

Consider using these exclusions:
[io.replikativ/datahike "0.6.1544" :exclusions [com.taoensso/timbre]]

[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/superv.async "0.3.43"]
 overrides
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/konserve "0.7.311"] -> [io.replikativ/superv.async "0.3.46"]

Consider using these exclusions:
[io.replikativ/datahike "0.6.1544" :exclusions [io.replikativ/superv.async]]

[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/incognito "0.3.66"] -> [fress "0.3.3"]
 overrides
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/hitchhiker-tree "0.2.222"] -> [io.replikativ/konserve "0.7.271"] -> [fress "0.4.0"]
 and
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/konserve "0.7.311"] -> [fress "0.4.0"]

Consider using these exclusions:
[io.replikativ/datahike "0.6.1544" :exclusions [fress]]

[io.replikativ/datahike "0.6.1544"] -> [com.taoensso/timbre "5.2.1"] -> [com.taoensso/encore "3.21.0"]
 overrides
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/konserve "0.7.311"] -> [com.taoensso/timbre "6.0.1"] -> [com.taoensso/encore "3.31.0"]

Consider using these exclusions:
[io.replikativ/datahike "0.6.1544" :exclusions [com.taoensso/encore]]

[io.replikativ/datahike "0.6.1544"] -> [com.taoensso/timbre "5.2.1"] -> [io.aviso/pretty "1.1.1"]
 overrides
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/konserve "0.7.311"] -> [com.taoensso/timbre "6.0.1"] -> [io.aviso/pretty "1.3"]

Consider using these exclusions:
[io.replikativ/datahike "0.6.1544" :exclusions [io.aviso/pretty]]

[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/superv.async "0.3.43"] -> [org.clojure/core.async "1.5.648"]
 overrides
[io.replikativ/datahike "0.6.1544"] -> [io.replikativ/konserve "0.7.311"] -> [io.replikativ/superv.async "0.3.46"] -> [org.clojure/core.async "1.6.673"]

Consider using these exclusions:
[io.replikativ/datahike "0.6.1544" :exclusions [org.clojure/core.async]]

 [io.replikativ/datahike "0.6.1544"]
   [com.taoensso/timbre "5.2.1"]
     [com.taoensso/encore "3.21.0"]
       [com.taoensso/truss "1.6.0"]
     [io.aviso/pretty "1.1.1"]
   [environ "1.2.0"]
   [io.replikativ/datalog-parser "0.2.25"]
   [io.replikativ/hasch "0.3.94"]
   [io.replikativ/hitchhiker-tree "0.2.222"]
     [com.taoensso/carmine "3.1.0"]
       [com.taoensso/nippy "3.1.1"]
         [org.iq80.snappy/snappy "0.4"]
         [org.tukaani/xz "1.8"]
       [commons-codec "1.15"]
       [org.apache.commons/commons-pool2 "2.9.0"]
     [org.clojure/core.cache "1.0.225"]
       [org.clojure/data.priority-map "1.1.0"]
     [org.clojure/core.memoize "1.0.257"]
     [org.clojure/core.rrb-vector "0.1.2"]
   [io.replikativ/incognito "0.3.66"]
     [com.cognitect/transit-clj "1.0.329"]
     [com.cognitect/transit-cljs "0.8.269"]
       [com.cognitect/transit-js "0.8.874"]
     [fress "0.3.3"]
     [org.clojure/data.fressian "1.0.0"]
       [org.fressian/fressian "0.6.6"]
     [org.clojure/tools.cli "1.0.206"]
   [io.replikativ/konserve "0.7.311"]
     [com.github.pkpkpk/cljs-node-io "2.0.332"]
     [io.replikativ/geheimnis "0.1.1"]
       [org.clojure/data.codec "0.1.0"]
       [org.clojure/java.classpath "0.2.3"]
     [org.clojars.mmb90/cljs-cache "0.1.4"]
       [adzerk/boot-test "1.1.2"]
       [crisptrutski/boot-cljs-test "0.2.2-20160402.204547-3"]
       [tailrecursion/cljs-priority-map "1.2.1"]
     [org.lz4/lz4-java "1.8.0"]
   [io.replikativ/superv.async "0.3.43"]
     [org.clojure/core.async "1.5.648"]
       [org.clojure/tools.analyzer.jvm "1.2.2"]
         [org.clojure/tools.analyzer "1.1.0"]
         [org.ow2.asm/asm "9.2"]
   [io.replikativ/zufall "0.2.9"]
   [junit "4.13.2"]
     [org.hamcrest/hamcrest-core "1.3"]
   [medley "1.4.0"]
   [metosin/spec-tools "0.10.5"]
   [mvxcvi/clj-cbor "1.1.1"]
   [org.clojure/clojurescript "1.11.60"]
     [com.cognitect/transit-java "1.0.362" :exclusions [[org.clojure/clojure]]]
       [com.fasterxml.jackson.core/jackson-core "2.8.7"]
       [javax.xml.bind/jaxb-api "2.3.0"]
       [org.msgpack/msgpack "0.6.12"]
         [com.googlecode.json-simple/json-simple "1.1.1" :exclusions [[junit]]]
         [org.javassist/javassist "3.18.1-GA"]
     [com.google.javascript/closure-compiler-unshaded "v20220502"]
     [org.clojure/google-closure-library "0.0-20211011-0726fdeb"]
       [org.clojure/google-closure-library-third-party "0.0-20211011-0726fdeb"]
     [org.clojure/tools.reader "1.3.6"]
   [persistent-sorted-set "0.2.3"]
 [nrepl "1.0.0" :exclusions [[org.clojure/clojure]]]
 [org.clojure/clojure "1.11.1"]
   [org.clojure/core.specs.alpha "0.2.62"]
   [org.clojure/spec.alpha "0.3.218"]
 [org.nrepl/incomplete "0.1.0" :exclusions [[org.clojure/clojure]]]
whilo commented 1 year ago

Thanks for taking the time to point this out. @TimoKramer @jsmassa are you aware of a way to handle this with deps? I think we discussed it at some point.

TimoKramer commented 1 year ago

with deps we have to use aliases and @jsmassa was already starting to take things apart but the PRs stalled. we have to go through all the replikativ-libs to resolve this issue.

rome-user commented 1 year ago

Although this is manual work, I think deps.edn lets you create a pom.xml file. Then you can edit the scope of dependencies in there. The downside is that you now have to maintain two sets of dependencies: those declared in deps.edn and those declared in pom.xml

whilo commented 1 year ago

Editing pom.xml files separately is really not ideal. Aliases seem to be the official way https://clojureverse.org/t/how-to-provide-a-dependency-scope-in-tools-deps/1862. But I guess there are problems for downstream users with this approach, I don't know how Maven users would do this for instance(?) We can also fix any of the other replikativ libs.

@rome-user I agree that Datahike should be as lean as possible and it actually does not require more than JDK, Clojure and core.async, some serialization libs + replikativ libs atm. Are you running in specific obstacles?

rome-user commented 1 year ago

No specific obstacles besides uberjars being very large (as they will inevitably bundle CLJS and related libraries), and the dependency tree having a lot of conflicts.

TimoKramer commented 11 months ago

using exclusions for now

TimoKramer commented 11 months ago

The latest release https://github.com/replikativ/datahike/releases/tag/0.6.1549 has the clojurescript-dependency removed. @rome-user