How to change lein-sub project to lein-modules, sub module has Cyclic dependency.

[jiyouyou125]

How to deal with Cyclic dependency with sub modules ?

java.lang.Exception: Cyclic dependency: cascalog-core, cascalog-checkpoint, midje-cascalog, cascalog-lzo, cascalog-more-taps, cascalog-math

https://github.com/nathanmarz/cascalog/pull/273/files

[jiyouyou125]

Is a way to build Cyclic dependency modules(A,B) to temp module D, Then A dependent D, B dependent D. like "build-helper-maven-plugin"

[jiyouyou125]

or a flag to use the order of its :dirs vector, otherwise use dependency order.

[jcrossley3]

I would expect the :dirs vector to be used if set. If not, might indicate a bug. Can the cyclic dep not be removed?

[jiyouyou125]

yes. use checkout, shows Cyclic dependency. cascalog-core and midje-cascalog has Cyclic dependency. lein modules install fail.

root project.clj:

        :dirs ["cascalog-core"
               ;;"cascalog-checkpoint"
               ;;"cascalog-lzo"
               ;;"cascalog-more-taps"
               ;;"cascalog-math"
               "midje-cascalog"
               ]

cascalog-core project.clj :profiles {:dev {:dependencies [[cascalog/midje-cascalog :version]] :plugins [[lein-midje "3.1.3"]]

midje-cascalog project.clj: :profiles {:provided { :dependencies [[cascalog/cascalog-core :version]] }})

[jiyouyou125]

lein sub install, success. the same order, lein modules throw cyclic dep. seems not work

[jcrossley3]

Yeah, that's a tricky one. Looking at the code, lein-modules will always build in the correct dependency order, whether :dirs is set or not. In other words, unlike lein-sub, the order of :dirs doesn't matter. I'm not sure I want to change that behavior. If it's important that core->midje->core, then I'm not sure lein-modules is right for that project.

I'm not familiar with build-helper-maven-plugin. How does it address cyclic deps?

[jiyouyou125]

cascalog-core only dependent midjie-cascalog when develop(:dev) scope. midjie-cascalog dependent cascalog-core in :provided scope.

could ignore :dev dependent ?

https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

[jiyouyou125]

build jar, it's not exactly cyclic dep.

[jcrossley3]

I'm not sure I understand the question. I could see maybe ignoring midje-cascalog's dependency on cascalog-core because it's :provided but then the risk is that midje-cascalog is built before cascalog-core.

[jiyouyou125]

ignore cascalog-core 's :dev { midjie-cascalog}

[jcrossley3]

Yes, that would work for your project, but what is the general rule we're applying that makes it a reasonable thing to do for all projects? Right now, the general rule is "cyclic dependencies are not allowed."

[jiyouyou125]

Yes. ^_^，allow :dir order also can resolve this situation。
:dev cyclic, i think it's not strict cyclic dependencies.

[jcrossley3]

To me, a far better solution is to either 1) not use midje to test cascalog-core or b) break out the stuff needed by both cascalog-core and midje-cascalog into its own module, say cascalog-api maybe. Either one of those solutions would enable you to eliminate :dirs entirely.

[jiyouyou125]

^_^, ok, thanks your advice. I'll try to do this.

when writing below, receive your advice.

I look the maven. It also distinguish scope. compile time don't need including test/dev scope's dependencies. test/dev has cycle situation, maybe other projects also suffer this.

From Maven:
/* * All dependencies that this project has, including transitive ones. Contents are lazily populated, so depending on * what phases have run dependencies in some scopes won't be included. eg. if only compile phase has run, * dependencies with scope test won't be included. * @return {@link Set} < {@link Artifact} > * @see #getDependencyArtifacts() to get only direct dependencies */

[jcrossley3]

Intuitively, I would expect lein with-profiles -provided modules to work. But it doesn't. However, I'm not sure that's a bug (at least, with lein-modules, I should say). Leiningen profile merging logic is notoriously unpredictable and brittle. Further, I don't believe the :provided profile in Leiningen is used the same way as in Maven. From the Leiningen docs,

"The :provided profile is used to specify dependencies that should be available during jar creation, but not propagated to other code that depends on your project. These are dependencies that the project assumes will be provided by whatever environment the jar is used in, but are needed during the development of the project. This is often used for frameworks like Hadoop that provide their own copies of certain libraries."

That "during jar creation" bit is what makes me think it might always be active during a build, but don't quote me on that. Have you tried putting the dep in another non-standard profile?

[jiyouyou125]

I tried lein with-profile and lein modules with-profile: result shows root project and modules project 's owner dependencies: affected by "with-profile"

only modules dependencies has no effect by "with-profile": i think it's a bug. modules relationship should affected by "with-profile".

lein version: Leiningen 2.5.3 on Java 1.7.0_51 Java HotSpot(TM) 64-Bit Server VM command: deps :tree

1) use ----- cd cascalog-core; a. lein with-profile provided deps :tree b. lein deps :tree dump the correct dependencies for cascalog-core. a. dependencies contains : provided and compile dependencies(top level dependencies). 2) checkout lein_modules branch. use -----
a. lein with-profile provided modules deps :tree b. lein modules deps :tree dump the correct dependencies for modules. a. dependencies contains: provided and compile dependencies(top level dependencies).