test step requires all artifacts to be present in M2

[gimesketvirtadieni]

Hi

I am exploring functionality this plugin provides and stumbled on one issue: lein monolith with-all :in app-a compile is able to resolve all ‘local’ dependencies from the source, however lein monolith with-all :in app-a test requires 'local' dependencies AND selected project's artifacts to be present at least in the local m2:

user@pc:~/lein-monolith/example$ lein monolith with-all :in app-a test
Could not find artifact example:app-a:jar:0.5.0 in central (https://repo1.maven.org/maven2/)
Could not find artifact example:app-a:jar:0.5.0 in clojars (https://repo.clojars.org/)
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.

The same result is for jar, uberjar steps as well. I would expect all these steps would pick up ‘local’ dependencies from project tree without using m2 or external repositories. To demonstrate this functionality I use a sample project provided in this repo.

Is my understanding correct? What was your rationale behind such functionality?

Thank you in advance :)

[greglook]

Yes, your understanding is correct. Since lein-monolith is mostly a wrapper around normal leiningen tasks, those tasks use the dependencies as declared in each project. Those dependencies are resolved in the normal fashion, so even subprojects of the monorepo must be installed in order to be picked up locally.

In some cases like the with-all task it could potentially strip out dependencies that it knew would be available on the classpath, but it's simpler to reason about dependencies working the way they normally do. Changing how the projects get picked up would require a much deeper change to leiningen.

You could use checkout links after the initial lein monolith each install if you want hot code reloading or testing across your projects. See lein help monolith link for details.

[gimesketvirtadieni]

Thank you for coming back so quickly :)

It’s good to know that my understanding was correct as I am very new to clojure and leiningen (I used to work with Maven before).

I suspected that you rely mostly on leiningen functionality, however one thing still bothers me: for compilation task monolith plugin does resolve ‘local’ dependencies and manage to use them from the source tree without relying on M2.

My guess is that you create a dependency graph and just apply compile task in the dependency order. Is it correct? BTW, do you create a dependency graph yourself or leiningen exposes some API around it?

Once again, thank you in advance for your reply.

P.S. BTW incremental build functionality provided by monolith is awesome! I did not explore it thoroughly (like for example if changed lib triggers rebuilding all local dependencies it depends on), but it is the only plugin with such ‘must-have’ functionality for any bigger code base.

[greglook]

for compilation task monolith plugin does resolve ‘local’ dependencies and manage to use them from the source tree

In your lein monolith with-all ... example above, the with-all subtask combines all of your projects' source/resource/test paths together. That's what would let you "see" the current sources for a compilation or tests. Most of the time you probably don't want to use with-all - instead, the expectation is that you're either working in one project at a time or using each to iterate over them.

My guess is that you create a dependency graph and just apply compile task in the dependency order.

Yep! lein-monolith tracks the dependency relationships among all of the subprojects, so that it can iterate over them in dependency order. Leiningen itself has no notion of multiple projects, so that's functionality built into the plugin.

BTW incremental build functionality provided by monolith is awesome! I did not explore it thoroughly

Thanks! It's super helpful in large monorepos - our production app has ~210 subprojects in it right now, so being able to rebuild just the projects you need is important. This is pretty useful for making sure the project you're working in is up-to-date:

lein monolith each :upstream :refresh build :parallel 4 install

This is good for testing all projects which depend on the one you're in, to make sure your changes haven't broken anything:

lein monolith each :downstream :refresh test do clean, check, test, install