zcaudate-me / lein-repack

moved -> 'lucid.distribute'
65 stars 5 forks source link

lein-repack

Repack your project for deployment and distribution

Whats New

0.2.10

0.2.9

0.2.8

0.2.7

0.2.4

0.2.3

0.1.4

0.1.3

Motivation

lein-repack was written to solve a problem I had with utilities and general purpose libraries. In my experience, clojure libraries are much better when they are distributed in small packages. However, functionality is much easier to develop when the project is big: when we try to build our general purpose libaries in small packages, they become a nightmare to test and deploy.

lein-repack redistributes your code base into managable, deployable chunks and analyses your source files to automatically resolve internal and external dependencies. It this way, a big clojure project can now be broken up into sub-packages for deployment and redistribution.

The plugin will:

Installation

Add lein-repack to your project.clj file that you wish to repack:

  (defproject lama ....

    :profiles {:dev {:plugins [...
                               [lein-repack "0.2.8"] ;; Current latest version
                               ...]}}

    ;; specify source folders for repack, defaults can be omitted
    :repack [{:type :clojure
              :path "src"
              :levels 1}])

Walkthrough

For example, if there were a project called lama at version 0.1.0 and the files were organised like this:

- src
   - lama
       - weapons
          - gun.clj
       - food
          - prepare.clj
       - food.clj
       - core.clj

Running lein repack install will split the project into four jars and install them in maven.

lama-0.1.0.jar
lama-core-0.1.0.jar
lama-food-0.1.0.jar
lama-weapons-0.1.0.jar

Running lein repack deploy will deploy all four artifacts to clojars.

Once the artifacts are installed/deployed, they are now ready to be used. For example, if only the functionality for lama.weapons were required for another project, it can be imported individually in the project by adding [lama.weapons "0.1.0"] to project dependencies. The entire project can be imported my adding [lama "0.1.0"] to project dependencies.

Usage

Once installed in your project.clj (or globally via profile.clj)

$ lein repack

Sub-tasks for repackage are available:
help               Display this message
manifest           Generate the manifest for repacking jars
split              Splits the main project into several interim projects
clean              Removes the interim folder
install            Install all the interim projects to local repo
deploy             Deploys all the repackaged jars
push               Deployment the old-school way

configuration

It's quite a difficult task to attempt to split a big project into smaller pieces. There are lots of things to consider, especially with mixed clj/cljs projects - most of all, where are all the different files stored and where are they going to go? Also, there is a problem with dependencies. For example, a file in your source directory may refer to an image file in your resource directory. When the project is split, you'd typically want just the image file to be relocated and not the entire resource directory.

lein-repack tries to be as accomodating as possible to these endless possibilities. Of course, you have to be smart about doing so but the tool does offer enough options to split up more intricate projects into bite-sized, more easily distributable pieces.

To be repacked, a project requires either a map or a vector of maps defining the code to be distributed under the :repack key in it's project.clj. We can look at options for repack.advance and how one project will get sliced up into multiple projects. The entire :repack options are shown below, with labels (#1 to #4) showing where relevent files are kept. In this case, we have a mixed source project with .clj, .cljs, .java and resources all split into their various directories. The final result is shown in the diagram below:

Repack Results

The configuration options that achieved such a result are shown below:

:repack [{:type :clojure
          :path "src/clj"         ;; # 1. Clojure Files
          :levels 2
          :standalone #{"web"}}
         {:type :clojure
          :path "src/cljs"        ;; # 2. Clojurescript Files
          :levels 2
          :standalone #{"web"}}
         {:subpackage "jvm"       ;; # 3. Java Files
          :path "java/im/chit/repack"
          :distribute {"common" #{"common"}
                       "web"    #{"web"}}}
         {:subpackage "resources" ;; # 4. Resource Files
          :path "resources"
          :distribute {"common" #{"common"}
                       "web"    #{"web"}}
          :dependents #{"core"}}]

We look at the first entry (#1) and see that it has keys :path, :levels and :standalone.

The second entry (#2) is the same as the first, except that the :path key points to src/cljs.

The third entry (#3) is a little different to the first two because we are defining a subpackage as well as rules for how subfolders have to be distributed across modules. In this case, we have a folder containing *.java files.

The forth entry (#4) is the resource directory and it is very similar to the java source code directory. However, some submodules may depend on resources and so this must be explicitly stated.

manifest

So having attempted to explain what was happening with the configuration options, it's probably much easier to just run the thing. The manifest will show what files in the project will go where depending on how the options play out. The advanced features of lein-repack will be demonstrated to show how different types of files (java, clj, cljs as well as resource files) can be developed together and then packaged seperately depending on project specification:

$ cd example/repack.advanced
$ lein repack manifest

The output the project manifest for the root project as well as all its branches can be seen below. Note the different types of files in the web branch:

{:root
 {:files [],
  :dependencies
  [[org.clojure/clojure "1.6.0"]
   [im.chit/korra "0.1.2"]
   [im.chit/repack.advance.resources "0.1.0-SNAPSHOT"]
   [im.chit/repack.advance.jvm "0.1.0-SNAPSHOT"]
   [im.chit/repack.advance.common "0.1.0-SNAPSHOT"]
   [im.chit/repack.advance.core "0.1.0-SNAPSHOT"]
   [im.chit/repack.advance.util.array "0.1.0-SNAPSHOT"]
   [im.chit/repack.advance.util.data "0.1.0-SNAPSHOT"]
   [im.chit/repack.advance.web "0.1.0-SNAPSHOT"]],
  :version "0.1.0-SNAPSHOT",
  :group "im.chit",
  :name "repack.advance"},
 :branches
 {"web"
  {:coordinate [im.chit/repack.advance.web "0.1.0-SNAPSHOT"],
   :files
   ["java/im/chit/repack/web/Client.java"
    "resources/web/a.html"
    "src/cljs/repack/web/client.cljs"
    "src/clj/repack/web/client.clj"
    "src/cljs/repack/web.cljs"
    "resources/web/b.html"
    "src/clj/repack/web.clj"],
   :dependencies
   [[org.clojure/clojure "1.6.0"]
    [im.chit/repack.advance.core "0.1.0-SNAPSHOT"]
    [im.chit/repack.advance.util.array "0.1.0-SNAPSHOT"]
    [im.chit/repack.advance.common "0.1.0-SNAPSHOT"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.web",
   :group "im.chit"},
  "util.data"
  {:coordinate [im.chit/repack.advance.util.data "0.1.0-SNAPSHOT"],
   :files ["src/clj/repack/util/data.clj"],
   :dependencies
   [[org.clojure/clojure "1.6.0"]
    [im.chit/repack.advance.util.array "0.1.0-SNAPSHOT"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.util.data",
   :group "im.chit"},
  "util.array"
  {:coordinate [im.chit/repack.advance.util.array "0.1.0-SNAPSHOT"],
   :files
   ["src/clj/repack/util/array.clj"
    "src/clj/repack/util/array/sort.clj"],
   :dependencies [[org.clojure/clojure "1.6.0"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.util.array",
   :group "im.chit"},
  "core"
  {:coordinate [im.chit/repack.advance.core "0.1.0-SNAPSHOT"],
   :files ["src/clj/repack/core.clj"],
   :dependencies
   [[org.clojure/clojure "1.6.0"]
    [im.chit/repack.advance.resources "0.1.0-SNAPSHOT"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.core",
   :group "im.chit"},
  "common"
  {:coordinate [im.chit/repack.advance.common "0.1.0-SNAPSHOT"],
   :files
   ["resources/common/a.txt"
    "java/im/chit/repack/common/Hello.java"
    "resources/common/b.txt"
    "src/clj/repack/common.clj"],
   :dependencies [[org.clojure/clojure "1.6.0"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.common",
   :group "im.chit"},
  "jvm"
  {:coordinate [im.chit/repack.advance.jvm "0.1.0-SNAPSHOT"],
   :files ["java/im/chit/repack/native/Utils.java"],
   :dependencies [[org.clojure/clojure "1.6.0"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.jvm",
   :group "im.chit"},
  "resources"
  {:coordinate [im.chit/repack.advance.resources "0.1.0-SNAPSHOT"],
   :files ["resources/stuff/y.edn" "resources/stuff/x.edn"],
   :dependencies [[org.clojure/clojure "1.6.0"]],
   :version "0.1.0-SNAPSHOT",
   :name "im.chit/repack.advance.resources",
   :group "im.chit"}}}

split

Running lein repack split will generate scaffolding of the root and branch projects in an interim directory, typically located in target/interim of your project directory.

clean, install, deploy

They work the same way as the built in leiningen commands but also generates subprojects

Contributors

License

Copyright © 2015 Chris Zheng

Distributed under the MIT License