ccw-ide / ccw

Counterclockwise is an Eclipse plugin helping developers write Clojure code
https://github.com/laurentpetit/ccw/wiki/GoogleCodeHome
Eclipse Public License 1.0
217 stars 49 forks source link

Multiple projects lein dependencies not compounded in the class path ? #497

Closed laurentpetit closed 9 years ago

laurentpetit commented 9 years ago

Original issue 456 created by laurentpetit on 2012-10-08T19:55:01.000Z:

Hi,

we run the beta 0.10.2.20121005.

here we have a "master" project (HIGIEBUSSupervisor) with several other projects as dependencies. The master project refers to the other projects as usual, using the project tab in the master project build path panel.

When we run lein externally (either lein run or lein repl), all the dependencies for all the sub-projects are present.

However when running the master project in ccw, not all the dependencies show up.

Here's what we have from the CCW repl

=> (pprint (sort (clojure.string/split (System/getProperty "java.class.path") #":"))) nil ("/home/lprefontaine/.m2/repository/avalon-framework/avalon-framework/4.1.3/avalon-framework-4.1.3.jar" "/home/lprefontaine/.m2/repository/avout/avout/0.5.3/avout-0.5.3.jar" "/home/lprefontaine/.m2/repository/clj-logging-config/clj-logging-config/1.9.8/clj-logging-config-1.9.8.jar" "/home/lprefontaine/.m2/repository/com/taoensso/nippy/0.10.1/nippy-0.10.1.jar" "/home/lprefontaine/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar" "/home/lprefontaine/.m2/repository/commons-codec/commons-codec/1.5/commons-codec-1.5.jar" "/home/lprefontaine/.m2/repository/commons-io/commons-io/1.4/commons-io-1.4.jar" "/home/lprefontaine/.m2/repository/commons-logging/commons-logging/1.1/commons-logging-1.1.jar" "/home/lprefontaine/.m2/repository/javax/servlet/servlet-api/2.3/servlet-api-2.3.jar" "/home/lprefontaine/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar" "/home/lprefontaine/.m2/repository/junit/junit/3.8.1/junit-3.8.1.jar" "/home/lprefontaine/.m2/repository/log4j/log4j/1.2.16/log4j-1.2.16.jar" "/home/lprefontaine/.m2/repository/logkit/logkit/1.0.1/logkit-1.0.1.jar" "/home/lprefontaine/.m2/repository/mysql/mysql-connector-java/5.1.16/mysql-connector-java-5.1.16.jar" "/home/lprefontaine/.m2/repository/net/sf/cron4j/cron4j/2.2.3/cron4j-2.2.3.jar" "/home/lprefontaine/.m2/repository/net/sourceforge/jtds/jtds/1.2.5/jtds-1.2.5.jar" "/home/lprefontaine/.m2/repository/ojdbc/ojdbc/14/ojdbc-14.jar" "/home/lprefontaine/.m2/repository/oraclepki/oraclepki/10.2/oraclepki-10.2.jar" "/home/lprefontaine/.m2/repository/orai18n/orai18n/10.2.0.3/orai18n-10.2.0.3.jar" "/home/lprefontaine/.m2/repository/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar" "/home/lprefontaine/.m2/repository/org/apache/zookeeper/zookeeper/3.3.2/zookeeper-3.3.2.jar" "/home/lprefontaine/.m2/repository/org/clojure/clojure/1.4.0/clojure-1.4.0.jar" "/home/lprefontaine/.m2/repository/org/clojure/java.jdbc/0.2.3/java.jdbc-0.2.3.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.cli/0.2.1/tools.cli-0.2.1.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.logging/0.2.3/tools.logging-0.2.3.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.nrepl/0.2.0-beta7/tools.nrepl-0.2.0-beta7.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.trace/0.7.3/tools.trace-0.7.3.jar" "/home/lprefontaine/.m2/repository/org/jasypt/jasypt/1.9.0/jasypt-1.9.0.jar" "/home/lprefontaine/.m2/repository/org/springframework/spring/2.5.5/spring-2.5.5.jar" "/home/lprefontaine/.m2/repository/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar" "/home/lprefontaine/.m2/repository/zookeeper-clj/zookeeper-clj/0.9.2/zookeeper-clj-0.9.2.jar" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSAuditAdapter/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSAuditDBModel/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSDbAdapters/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSHMSAdapters/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSHMSAdapters/resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSProtocols/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSProtocols/resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSRegistryEditor/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSRouter/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSServices/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/dev-resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/src" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/test" "/home/lprefontaine/workspaces/higiebus/clj-record/bin" "/home/lprefontaine/workspaces/higiebus/clj-record/target/classes" "/opt/clojure1.4/clojure-1.4.0.jar" "/opt/higiebus/apps/libraries/cron4j-2.2.3.jar")

While when using lein repl in the same master project:

higiebus.main.supervisor=> (pprint (sort (clojure.string/split (System/getProperty "java.class.path") #":"))) nil("/home/lprefontaine/.m2/repository/HIGIEBUSAuditAdapter/HIGIEBUSAuditAdapter/1.5.0/HIGIEBUSAuditAdapter-1.5.0.jar" "/home/lprefontaine/.m2/repository/HIGIEBUSAuditDBModel/HIGIEBUSAuditDBModel/1.5.0/HIGIEBUSAuditDBModel-1.5.0.jar" "/home/lprefontaine/.m2/repository/HIGIEBUSController/HIGIEBUSController/1.5.0/HIGIEBUSController-1.5.0.jar" "/home/lprefontaine/.m2/repository/HIGIEBUSHMSAdapters/HIGIEBUSHMSAdapters/1.5.0/HIGIEBUSHMSAdapters-1.5.0.jar" "/home/lprefontaine/.m2/repository/HIGIEBUSProtocols/HIGIEBUSProtocols/1.5.0/HIGIEBUSProtocols-1.5.0.jar" "/home/lprefontaine/.m2/repository/HIGIEBUSServices/HIGIEBUSServices/1.5.0/HIGIEBUSServices-1.5.0.jar" "/home/lprefontaine/.m2/repository/antlr/antlr/2.7.6/antlr-2.7.6.jar" "/home/lprefontaine/.m2/repository/asm/asm-attrs/1.5.3/asm-attrs-1.5.3.jar" "/home/lprefontaine/.m2/repository/asm/asm/1.5.3/asm-1.5.3.jar" "/home/lprefontaine/.m2/repository/avalon-framework/avalon-framework/4.1.3/avalon-framework-4.1.3.jar" "/home/lprefontaine/.m2/repository/avout/avout/0.5.3/avout-0.5.3.jar" "/home/lprefontaine/.m2/repository/backport-util-concurrent/backport-util-concurrent/3.1/backport-util-concurrent-3.1.jar" "/home/lprefontaine/.m2/repository/boing/boing/1.3.1/boing-1.3.1.jar" "/home/lprefontaine/.m2/repository/bouncycastle/bcmail-jdk14/138/bcmail-jdk14-138.jar" "/home/lprefontaine/.m2/repository/bouncycastle/bcprov-jdk14/138/bcprov-jdk14-138.jar" "/home/lprefontaine/.m2/repository/cglib/cglib/2.1_3/cglib-2.1_3.jar" "/home/lprefontaine/.m2/repository/cheshire/cheshire/3.1.0/cheshire-3.1.0.jar" "/home/lprefontaine/.m2/repository/clj-http/clj-http/0.3.5/clj-http-0.3.5.jar" "/home/lprefontaine/.m2/repository/clj-logging-config/clj-logging-config/1.9.8/clj-logging-config-1.9.8.jar" "/home/lprefontaine/.m2/repository/clj-pdf/clj-pdf/0.9.9/clj-pdf-0.9.9.jar" "/home/lprefontaine/.m2/repository/clj-record/clj-record/1.1.1/clj-record-1.1.1.jar" "/home/lprefontaine/.m2/repository/clojure-complete/clojure-complete/0.2.1/clojure-complete-0.2.1.jar" "/home/lprefontaine/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.0.0/jackson-core-2.0.0.jar" "/home/lprefontaine/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.0.0/jackson-dataformat-smile-2.0.0.jar" "/home/lprefontaine/.m2/repository/com/lowagie/itext/2.1.7/itext-2.1.7.jar" "/home/lprefontaine/.m2/repository/com/taoensso/nippy/0.10.1/nippy-0.10.1.jar" "/home/lprefontaine/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar" "/home/lprefontaine/.m2/repository/commons-codec/commons-codec/1.5/commons-codec-1.5.jar" "/home/lprefontaine/.m2/repository/commons-collections/commons-collections/2.1.1/commons-collections-2.1.1.jar" "/home/lprefontaine/.m2/repository/commons-dbcp/commons-dbcp/1.2.2/commons-dbcp-1.2.2.jar" "/home/lprefontaine/.m2/repository/commons-io/commons-io/1.4/commons-io-1.4.jar" "/home/lprefontaine/.m2/repository/commons-logging/commons-logging/1.1/commons-logging-1.1.jar" "/home/lprefontaine/.m2/repository/commons-pool/commons-pool/1.3/commons-pool-1.3.jar" "/home/lprefontaine/.m2/repository/dom4j/dom4j/1.6.1/dom4j-1.6.1.jar" "/home/lprefontaine/.m2/repository/javax/servlet/servlet-api/2.3/servlet-api-2.3.jar" "/home/lprefontaine/.m2/repository/javax/transaction/jta/1.0.1B/jta-1.0.1B.jar" "/home/lprefontaine/.m2/repository/jfree/jcommon/1.0.16/jcommon-1.0.16.jar" "/home/lprefontaine/.m2/repository/jfree/jfreechart/1.0.13/jfreechart-1.0.13.jar" "/home/lprefontaine/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar" "/home/lprefontaine/.m2/repository/joda-time/joda-time/1.5.1/joda-time-1.5.1.jar" "/home/lprefontaine/.m2/repository/junit/junit/4.8.1/junit-4.8.1.jar" "/home/lprefontaine/.m2/repository/log4j/log4j/1.2.16/log4j-1.2.16.jar" "/home/lprefontaine/.m2/repository/logkit/logkit/1.0.1/logkit-1.0.1.jar" "/home/lprefontaine/.m2/repository/mysql/mysql-connector-java/5.1.16/mysql-connector-java-5.1.16.jar" "/home/lprefontaine/.m2/repository/net/sf/cron4j/cron4j/2.2.3/cron4j-2.2.3.jar" "/home/lprefontaine/.m2/repository/net/sf/ehcache/ehcache/1.4.1/ehcache-1.4.1.jar" "/home/lprefontaine/.m2/repository/net/sf/jsr107cache/jsr107cache/1.0/jsr107cache-1.0.jar" "/home/lprefontaine/.m2/repository/net/sourceforge/jtds/jtds/1.2.5/jtds-1.2.5.jar" "/home/lprefontaine/.m2/repository/ojdbc/ojdbc/14/ojdbc-14.jar" "/home/lprefontaine/.m2/repository/oraclepki/oraclepki/10.2/oraclepki-10.2.jar" "/home/lprefontaine/.m2/repository/orai18n/orai18n/10.2.0.3/orai18n-10.2.0.3.jar" "/home/lprefontaine/.m2/repository/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar" "/home/lprefontaine/.m2/repository/org/apache/httpcomponents/httpclient/4.1.2/httpclient-4.1.2.jar" "/home/lprefontaine/.m2/repository/org/apache/httpcomponents/httpcore/4.1.2/httpcore-4.1.2.jar" "/home/lprefontaine/.m2/repository/org/apache/httpcomponents/httpmime/4.1.2/httpmime-4.1.2.jar" "/home/lprefontaine/.m2/repository/org/apache/zookeeper/zookeeper/3.3.2/zookeeper-3.3.2.jar" "/home/lprefontaine/.m2/repository/org/bouncycastle/bcmail-jdk14/1.38/bcmail-jdk14-1.38.jar" "/home/lprefontaine/.m2/repository/org/bouncycastle/bcprov-jdk14/1.38/bcprov-jdk14-1.38.jar" "/home/lprefontaine/.m2/repository/org/bouncycastle/bctsp-jdk14/1.38/bctsp-jdk14-1.38.jar" "/home/lprefontaine/.m2/repository/org/clojure/clojure/1.4.0/clojure-1.4.0.jar" "/home/lprefontaine/.m2/repository/org/clojure/core.incubator/0.1.0/core.incubator-0.1.0.jar" "/home/lprefontaine/.m2/repository/org/clojure/data.codec/0.1.0/data.codec-0.1.0.jar" "/home/lprefontaine/.m2/repository/org/clojure/java.jdbc/0.2.3/java.jdbc-0.2.3.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.cli/0.2.1/tools.cli-0.2.1.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.logging/0.2.3/tools.logging-0.2.3.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.nrepl/0.2.0-beta7/tools.nrepl-0.2.0-beta7.jar" "/home/lprefontaine/.m2/repository/org/clojure/tools.trace/0.7.3/tools.trace-0.7.3.jar" "/home/lprefontaine/.m2/repository/org/hibernate/hibernate/3.2.5.ga/hibernate-3.2.5.ga.jar" "/home/lprefontaine/.m2/repository/org/jasypt/jasypt/1.9.0/jasypt-1.9.0.jar" "/home/lprefontaine/.m2/repository/org/springframework/spring/2.5.5/spring-2.5.5.jar" "/home/lprefontaine/.m2/repository/org/thnetos/cd-client/0.3.4/cd-client-0.3.4.jar" "/home/lprefontaine/.m2/repository/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar" "/home/lprefontaine/.m2/repository/slingshot/slingshot/0.10.2/slingshot-0.10.2.jar" "/home/lprefontaine/.m2/repository/xml-apis/xml-apis/1.0.b2/xml-apis-1.0.b2.jar" "/home/lprefontaine/.m2/repository/zookeeper-clj/zookeeper-clj/0.9.2/zookeeper-clj-0.9.2.jar" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/classes" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/dev-resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/dev-resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/resources" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/src" "/home/lprefontaine/workspaces/higiebus/HIGIEBUSSupervisor/test")

As an example, jfree libs are not present in CCW. They are specified in the project.clj file of the HIGIEBUSProtocols project.

All the dire lein dependencies from the project.clj file in the master project are there in both cases.

Note that there is a potential conflict, the project.clj file of the master projects refers to artifacts (HIGIEBUS*.jar) files of each of the sub projects. hat may well be a source of conflict later at execution time. However, they do not appear in the nrepl java class path anyway.

Maybe the conflict is detected somehow in CCW at class path build time and the entire dependency set of sub projects is discarded ?

Meanwhile we added a user lib with a bunch of jars to make sure we are not missing anything. However we would like to keep dependencies close to the project were they are needed. The master project here does not refer directly to any of the jfree libs. Adding these at the top is redundant and needs to be managed manually.

In production we ship a uberjar to avoid manual management of these.

I attached the HIGIEBUSSupervisor dependency file and one of the sub project (HIGIEBUSProtocols).

Thank you, Luc

laurentpetit commented 9 years ago

Comment #1 originally posted by laurentpetit on 2012-10-08T20:43:58.000Z:

I think I understand: it seems that, right now, the "Leinigen Dependencies Classpath Container" is not marked as "exported to other projects". I'll try to fix that in the sources. In the mean time, you can try to fix this manually for your projects via the java build path > Order and Export Tab.

laurentpetit commented 9 years ago

Comment #2 originally posted by laurentpetit on 2012-10-08T20:57:59.000Z:

<empty>

laurentpetit commented 9 years ago

Comment #3 originally posted by laurentpetit on 2012-10-08T21:43:03.000Z:

I tried the export feature, it worked halfway after doing a project cleanup/rebuild. When I expand the Leiningen Dependencies node in project explorer in the master project, I see all of them now.

However, now it fails saying that some Joda class is not found. The class is visible in the Leiningen Dependencies node when I expand the Joda jar file.

In the repl, I have the same restricted set of dependencies as reported in this issue.

Anything else I should try ?

laurentpetit commented 9 years ago

Comment #4 originally posted by laurentpetit on 2012-10-10T01:54:06.000Z:

I investigated a bit. When I clean and rebuild projects in CCW, I get only .clj files in the classes folder.

The external build via lein generates only class files (all project files specify AOT and no sources in the targets).

The failure occurs after a rebuild from ccw, the compiler attempts to compile the .clj file which references Joda and does not find Joda in the class path of the REPL.

There are no errors reported by the internal rebuild. However an external build followed by a refresh in project explorer generates a few errors regarding the global state of the projects but no details are provided as to what triggered it.

This is the best I can find to illustrate the behaviour difference between a CCW internal build and an external lein build.

When I examine the list of dependencies in the REPL, I found that some libs in a few of the dependent project is present while others are absent.

Kind of weird. I would have expected none of these to show up.

The list is complete in the Leiningen dependencies node in the project explorer of the master project, including joda which does not show up at compile time in the class path.

I doubled check the export flag and it set on the lein dependency node in all the projects on which the master project depends. This seems to corroborate what I see in the Leiningen dependencies node.

Is there a way I can investigate further down in Eclipse or is the problem considered solved even if my manual attempts failed ?

Thank you,

laurentpetit commented 9 years ago

Comment #5 originally posted by laurentpetit on 2012-10-10T03:21:34.000Z:

Another update.

I parsed the HIGIEBUSSupervisor.container to create a copy script, dumped all the dependencies in a folder, added all the jars to the classpath of the master project except the ones generated by the sub-targets (HIGIEBUSxxx.jar).

A noticeable improvement of the internal build, no compiler errors on a clj file, it seems to find the java classes it missed before. However we have some gen-class directives that do not trigger a .class file generation. So these are missing and the compiler eventually fails with a missing java class (from the genclass directive).

The class path in the REPL looks ok if I compare it to the master project leiningen dependencies node in project explorer.

From now on I will use an external lein build but will be able to run the master project main in the REPL.

Thank you,

Eventually the compiler fails.

laurentpetit commented 9 years ago

Comment #6 originally posted by laurentpetit on 2012-10-10T19:56:48.000Z:

Wow, seems like a really nasty nasty bug we're facing :-(. Reopening the issue. Will have to work harder on this one ...

laurentpetit commented 9 years ago

Comment #7 originally posted by laurentpetit on 2012-11-10T21:44:38.000Z:

<empty>

laurentpetit commented 9 years ago

Comment #8 originally posted by laurentpetit on 2012-11-15T10:29:48.000Z:

<empty>

laurentpetit commented 9 years ago

Comment #9 originally posted by laurentpetit on 2014-05-22T07:44:15.000Z:

Sorry to not have been able to work on this. Too old report, please reopen or submit a new one if the issue still occurs with latest versions.