Feature request: a task doing only download dependencies

ailisp commented 5 years ago

Problem Description

A task just like lein deps and doesn't check any source/resource/asset files exist or not and other task options valid or not. This will be very helpful for build from dockerfile. Because if so, I can COPY project-path/build.boot . and install dependencies before doing COPY project-path . and, RUN boot production build target. Then if no changes happen to build.boot, docker can reuse the layer of download dependencies and the total docker build time is only the boot build time. In current case, I have toCOPY project-path . And any change in source will cause boot re-downloading all dependencies.

Steps to reproduce

Platform details

Platform (macOS, Linux, Windows): macOS Platform version: 10.13 JRE/JDK version (java -version):

java version "1.8.0_172"
Java(TM) SE Runtime Environment (build 1.8.0_172-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

Boot details

Boot version (2.7.1): 2.7.2 build.boot present? (yes/no): yes ~/.boot/profile present? (yes/no): no Task name? (if applicable): -

Please also provide the contents of build.boot and ~/.boot/profile (if applicable). Doesn't really matter here, give build.boot created by this as an example: boot -d boot/new new -t tenzing -n your-app -a +reagent -a +test

Thank you!

nha commented 5 years ago

nothing prevents you to make such a task. Something like:

(deftask deps [] clojure.core/identity)

(assuming you are already doing a set-env! in your build.boot)

ailisp commented 5 years ago

@nha Thanks! Just tried and works like a magic. Very clever solution

ailisp commented 5 years ago

Probably we still want an official builtin boot deps command, Although we have @nha's clear and brilliant solution. Because, if we have a trivial boot file

(set-env!
 :source-paths    #{"src/cljs" "src/clj"}
 :resource-paths  #{"resources"}
 :dependencies '[[adzerk/boot-cljs          "2.0.0"      :scope "test"]
                            ;; ...
                            [reagent "0.8.0"]])
(deftask deps [] clojure.core/identity)

Then if we

COPY ./build.boot . # copy build.boot
RUN boot deps # fail because no `src` dir there
COPY .  . # copy whole project
RUN boot prod build target

If we copy everything:

COPY .  .
boot deps
boot prod build target

In this case, any changes in src/ will cause layer COPY . . and layers after that can't be reused, and boot deps have to install all dependencies again.

Ideally boot deps can be a special task that doesn't require src, resource etc. exists. If so, the first example dockerfile snippet works, and when no changes in build.boot, dependencies download won't happen in further docker build, which accelerates a lot. lein deps works this way (try delete everything but project.clj, lein deps still works)

nha commented 5 years ago

I haven’t tested these, so just throwing ideas here.

possible solution 1:

remove set-env! from the global build.boot space, (perhaps into a function that tasks would call)
and do a different set-env! in the deps task and keep only what you want, like :dependencies

possible solution 2:

use an alternative build.boot file, say build.docker.boot (not sure if boot supports running a different file, but it would be easy with docker COPY)

possible solution 3:

create empty src/clj src/cljs directories in your Dockerfile

In any case, interested to see where this goes, particularly with building the target inside a Dockerfile I currently build it outside in bash and then COPY the final-uberjar.jar inside the docker image. So interested to know where you end up @ailisp :)

ailisp commented 5 years ago

@nha Thanks for the detailed discussion. I prefer solution 2, and I can rename it to build.docker.boot when copy it, and override it when copy whole project. And I think other two solutions definitely work. Uberjar locally works but not suitable for our current way of deployment. We're using openshift, which can only do docker build from a nearly fresh host (no jvm, only docker) so must have all build steps in dockerfile

nha commented 5 years ago

It looks like there is support for using a different build.boot file actually, using 'BUILD_BOOT' env var, which is arguable better:

ailisp commented 5 years ago

@nha Awesome find. Thank you so much

alexander-yakushev commented 5 years ago

@ailisp The way I do it is as @nha suggested:

COPY build.boot ./
RUN mkdir src/ test/ res/ && boot deps
...

It's hackish, yes. However, I think that overriding the Boot behavior of reacting to missing directories for one particular task and one particular usecase that involves another tool (Docker) is even worse.

ailisp commented 5 years ago

@alexander-yakushev Yes this probably the shortest and nice solution (@nha's solution 3). Thank you!

nha commented 5 years ago

(4) Yes another way to do it without adding a task:

# download & install deps, cache REPL and web deps
RUN /usr/bin/boot web -s doesnt/exist repl -e '(System/exit 0)' && rm -rf target

https://github.com/adzerk-oss/boot-clj-docker-image/blob/master/Dockerfile#L31-L32

boot-clj / boot