martinklepsch commented 9 years ago

There are some differences between the time Lein and Boot need to compile ClojureScript projects:

Optimizations :none

boot-cljs + :none 67sec
lein-cljsbuild + :none 39sec
Optimizations :advanced
boot-cljs + :advanced 240sec
lein-cljsbuild + :advanced 59sec
Incremental compiles
boot-cljs + :none 4sec
lein-cljsbuild + :none 0.6sec

Some of this might be unavoidable due to Boot's immutable fileset but there is probably still some space to optimize things.

(source)

Deraen commented 9 years ago

Some results from one of my projects.

Linux, i7-5820K, 32GB, SSD.

❯ cloc src 
     107 text files.
     107 unique files.                                          
       6 files ignored.

http://cloc.sourceforge.net v 1.60  T=0.31 s (328.8 files/s, 22720.9 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
LESS                            32            346            272           2649
ClojureScript                   43            307            109           2301
Clojure                         26            142             36            818
-------------------------------------------------------------------------------
SUM:                           101            795            417           5768
-------------------------------------------------------------------------------

Cold build once

❯ time boot cljs
Compiling ClojureScript...
• js/main.js
boot cljs  36,80s user 1,35s system 137% cpu 27,661 total

❯ time lein cljsbuild once dev
Compiling ClojureScript.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 18.679 seconds.
lein cljsbuild once dev  40,86s user 5,23s system 144% cpu 31,933 total

Cold build + incremental changes

❯ boot watch cljs --compiler-options '{:compiler-stats true}'

Starting file watcher (CTRL-C to quit)...

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 5091.174241 msecs
Add dependencies, elapsed time: 13638.943387 msecs
Elapsed time: 22.732 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 6.011365 msecs
Add dependencies, elapsed time: 3229.585955 msecs
Elapsed time: 4.117 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 5.79919 msecs
Add dependencies, elapsed time: 293.383856 msecs
Elapsed time: 0.691 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 5.564877 msecs
Add dependencies, elapsed time: 294.504514 msecs
Elapsed time: 0.725 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 5.868821 msecs
Add dependencies, elapsed time: 290.406285 msecs
Elapsed time: 0.650 sec

❯ lein cljsbuild auto dev
Compiling ClojureScript.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 6290.940972 msecs
Add dependencies, elapsed time: 10404.453076 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 16.833 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 150.461035 msecs
Add dependencies, elapsed time: 197.713174 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.386 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 120.344635 msecs
Add dependencies, elapsed time: 203.488199 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.353 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 134.10164 msecs
Add dependencies, elapsed time: 191.68372 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.352 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 164.868602 msecs
Add dependencies, elapsed time: 186.245324 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.378 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 108.110009 msecs
Add dependencies, elapsed time: 182.326096 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.318 seconds.

Advanced build

❯ time boot cljs -O advanced 
Compiling ClojureScript...
• js/main.js
boot cljs -O advanced  82,75s user 1,62s system 139% cpu 1:00,36 total

❯ time lein cljsbuild once adv
Compiling ClojureScript.
Compiling "target/cljsbuild-adv/public/js/main.js" from ["src/cljs" "src/cljc"]...
Successfully compiled "target/cljsbuild-adv/public/js/main.js" in 50.975 seconds.
lein cljsbuild once adv  85,62s user 7,84s system 146% cpu 1:03,74 total

Results:

Yes, boot is slower with incremental recompiles
- Cljs compiler takes similar amount of time in recompiles in both cases
- Difference between time taken by Cljs compiler and boot elapsed time is probably overhead from Boot
Boot is a bit faster WHEN taking account start up or boot or lein, and lein is very slow

Note: there is workaround in 1.7.48 versions which can cause long build times if there are many "additional" sources in source/resource-paths. There is a branch available without the workaround: https://github.com/adzerk-oss/boot-cljs/tree/no-workaround

jeluard commented 9 years ago

You might want to consider differences in environment too. I remember seeing significant slowdown with boot on OSX. Also it might be worth testing with tmpfs to reduce the file copy costs.

martinklepsch commented 9 years ago

I wonder if there are some large-ish projects that can be used as common ground for comparisons? Potential candidates:

Deraen commented 9 years ago

Unfortunately I'm testing with a work project which I can't share. I could probably use it as base for some public project sometime, but I'm quite busy for next month or so.

OS X tests coming in a bit.

Deraen commented 9 years ago

OS X, Retina 15" mid 2015, i7-4870HQ, 16GB, SSD.

Cold build once

❯ time boot cljs
Compiling ClojureScript...
• js/main.js
boot cljs  43,42s user 3,13s system 110% cpu 42,011 total

❯ time lein cljsbuild once dev
Compiling ClojureScript.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 23.025 seconds.
lein cljsbuild once dev  44,67s user 2,46s system 116% cpu 40,480 total

Cold build + incremental changes

❯ boot watch cljs --compiler-options '{:compiler-stats true}'

Starting file watcher (CTRL-C to quit)...

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 5532.328493 msecs
Add dependencies, elapsed time: 16878.173062 msecs
Elapsed time: 27,357 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 7.094487 msecs
Add dependencies, elapsed time: 3972.478977 msecs
Elapsed time: 5,057 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 3.683604 msecs
Add dependencies, elapsed time: 290.637781 msecs
Elapsed time: 0,852 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 3.696175 msecs
Add dependencies, elapsed time: 285.750784 msecs
Elapsed time: 0,899 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 3.576367 msecs
Add dependencies, elapsed time: 256.398092 msecs
Elapsed time: 0,840 sec

Compiling ClojureScript...
• js/main.js
Compile basic sources, elapsed time: 3.451776 msecs
Add dependencies, elapsed time: 269.087946 msecs
Elapsed time: 0,847 sec

❯ lein cljsbuild auto dev
Compiling ClojureScript.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 7497.729882 msecs
Add dependencies, elapsed time: 11892.622564 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 19.59 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 114.629484 msecs
Add dependencies, elapsed time: 222.701074 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.365 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 114.692085 msecs
Add dependencies, elapsed time: 199.180979 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.342 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 94.153035 msecs
Add dependencies, elapsed time: 211.069841 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.335 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 105.739917 msecs
Add dependencies, elapsed time: 198.917271 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.331 seconds.
Compiling "target/cljsbuild-dev/public/js/main.js" from ["src/cljs" "src/cljc"]...
Compile basic sources, elapsed time: 118.775211 msecs
Add dependencies, elapsed time: 193.77606 msecs
Successfully compiled "target/cljsbuild-dev/public/js/main.js" in 0.337 seconds.

Advanced build

❯ time boot cljs -O advanced
Compiling ClojureScript...
• js/main.js
boot cljs -O advanced  85,20s user 2,26s system 128% cpu 1:08,18 total

❯ time lein cljsbuild once adv
Compiling ClojureScript.
Compiling "target/cljsbuild-adv/public/js/main.js" from ["src/cljs" "src/cljc"]...
Successfully compiled "target/cljsbuild-adv/public/js/main.js" in 56.29 seconds.
lein cljsbuild once adv  80,26s user 2,76s system 117% cpu 1:10,73 total

Results:

Boot overhead seems to be a bit larger than on Linux

onetom commented 9 years ago

@martinklepsch @Deraen https://github.com/exicon/homepage/tree/SPA is a largish project IMHO.

(I'm merging this SPA branch back into master very soon. External scripts are giving me errors in :simple and :advanced optimization modes; that's my only obstacle now.)

tonsky commented 9 years ago

Is there some kind of verbose mode so I can share where time is spent? My env (initial build times in this post) is retina MBP 15" (Mid 2012), CLJS 1.7.28 if I remember correctly

Deraen commented 9 years ago

@tonsky For cljs compiler there is :compiler-stats true, but for Boot there is no such mode yet. We are investigating where the time is being spent and it's possible that such mode will be added.

We tested Boot with change that disabled writing files to target during build (https://github.com/boot-clj/boot/tree/disable-target-during-build) but that did not seem to have much affect.

Ps. We are discussing this at Boot channel on Clojurians Slack.

Deraen commented 9 years ago

@onetom Unfortunately hoplon in that project makes in unsuitable for comparing to Leiningen, afaik. Could still be useful for making sure if boot/boot-cljs changes are working.

ajchemist commented 9 years ago

In my case, I'm currently using boot-cljs at :advanced time only.

well in :none time, I'm using boot-figwheel written for my own purpose. of course, not fully compatible with boot task mental model :P but it will be helpful.

martinklepsch commented 9 years ago

TLDR; Slight differences when building once, incremental compiles similar. MacBook Air, i5, 8gb, no explicit JVM options for Lein or Boot. This is on a project where a friend said he gets constant 2s incremental compiles with cljsbuild and strong fluctuations between 4-11s with boot-cljs, his machine is a MacBook Pro so slightly stronger than mine.

cloc (not really informing imho)

Because cljs files in dependencies stats like this are not really informing I think. Extra definitions for cljc files.

$ cloc --read-lang-def=my_definitions.txt  src/
      19 text files.
      19 unique files.
       0 files ignored.

http://cloc.sourceforge.net v 1.64  T=0.15 s (127.3 files/s, 12082.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
ClojureScript                   17            247             60           1299
Clojure (cljc)                   1             52             16            105
Clojure                          1              3              1             20
-------------------------------------------------------------------------------
SUM:                            19            302             77           1424
-------------------------------------------------------------------------------

boot-cljs inital with cljs.edn

$ time boot  cljs -O none
Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 54765.245303 msecs
Add dependencies, elapsed time: 35700.274179 msecs
      112.42 real       212.66 user         7.79 sys

$ time boot  cljs -O none
Compiling ClojureScript...
• main.js
Add dependencies, elapsed time: 40620.430881 msecs
      115.62 real       223.61 user         7.26 sys

boot-cljs incremental with cljs.edn

Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 4667.451921 msecs
Add dependencies, elapsed time: 351.919653 msecs
Elapsed time: 6.113 sec

Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 4978.379127 msecs
Add dependencies, elapsed time: 347.62044 msecs
Elapsed time: 6.349 sec

Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 4363.361037 msecs
Add dependencies, elapsed time: 278.166531 msecs
Elapsed time: 5.564 sec

Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 4942.965445 msecs
Add dependencies, elapsed time: 307.56638 msecs
Elapsed time: 6.219 sec

Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 4420.814543 msecs
Add dependencies, elapsed time: 414.475259 msecs
Elapsed time: 5.695 sec

Compiling ClojureScript...
• main.js
Compile basic sources, elapsed time: 4451.585897 msecs
Add dependencies, elapsed time: 419.765311 msecs
Elapsed time: 6.007 sec

time lein cljsbuild once dev

$ rm -rf target/; time lein cljsbuild once dev
Compiling ClojureScript.
Compiling "target/out/main.js" from ["src"]...
Compile basic sources, elapsed time: 22618.106 msecs
Add dependencies, elapsed time: 33607.39 msecs
Successfully compiled "target/out/main.js" in 56.697 seconds.
       83.87 real       127.30 user         3.28 sys

lein cljsbuild auto dev

$ lein cljsbuild auto dev
Compiling ClojureScript.
Compiling "target/out/main.js" from ["src"]...
Compile basic sources, elapsed time: 1266.973 msecs
Add dependencies, elapsed time: 924.28 msecs
Successfully compiled "target/out/main.js" in 2.356 seconds.
Compiling "target/out/main.js" from ["src"]...
15:18:47.815 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider
15:18:48.450 [main] DEBUG c.a.internal.config.InternalConfig - Configuration override awssdk_config_override.json not found.
Compile basic sources, elapsed time: 23367.072 msecs
Add dependencies, elapsed time: 488.449 msecs
Successfully compiled "target/out/main.js" in 23.927 seconds.
Compiling "target/out/main.js" from ["src"]...
Compile basic sources, elapsed time: 5832.516 msecs
Add dependencies, elapsed time: 356.207 msecs
Successfully compiled "target/out/main.js" in 6.235 seconds.
Compiling "target/out/main.js" from ["src"]...
Compile basic sources, elapsed time: 5475.229 msecs
Add dependencies, elapsed time: 320.049 msecs
Successfully compiled "target/out/main.js" in 5.836 seconds.
Compiling "target/out/main.js" from ["src"]...
Compile basic sources, elapsed time: 6666.226 msecs
Add dependencies, elapsed time: 313.404 msecs
Successfully compiled "target/out/main.js" in 7.033 seconds.
Compiling "target/out/main.js" from ["src"]...
Compile basic sources, elapsed time: 5441.697 msecs
Add dependencies, elapsed time: 462.447 msecs
Successfully compiled "target/out/main.js" in 5.952 seconds.

piranha commented 9 years ago

That's strange, I should find some time and post my timings - for me difference is like 0.4-0.6s vs 1.6-2s with incremental compilation.

martinklepsch commented 9 years ago

@piranha is that figwheel or cljsbuild? I wonder if these two could have significant differences already?

piranha commented 9 years ago

figwheel, but I had an impression that there is no perceivable difference between figwheel and cljsbuild

Deraen commented 9 years ago

They are two completely different different plugins and there are notable differences.

martinklepsch commented 9 years ago

I feel like the only way going forward with this is creating a public project that has all three tools setup. Could even be used as a point of comparison by authors of figwheel and cljsbuild, potentially even some basic build.clj approach. cljs-bench. :smile:

Deraen commented 9 years ago

Planned Cljs compiler improvements should help (everyone) and looking into Boot tempdir/fileset performance should be useful as it's evident it's causing relatively large overhead in incremental compile. I would guess the overhead is pretty constant for each change and depends on numer of changed files or number of files in fileset.

onetom commented 9 years ago

I've narrowed my performance issue to this:

    [adzerk/boot-cljs "0.0-3308-0"]
    [org.clojure/clojurescript "0.0-3308"] ; or even "1.7.48", it makes no difference

Elapsed time: 17.847 sec (initial compile)
Elapsed time: 3.442 sec (1st incremental compile)
Elapsed time: 1.935 sec (2nd incremental compile)
Elapsed time: 1.907 sec
Elapsed time: 1.902 sec

vs

    [adzerk/boot-cljs "1.7.48-0"]
    [org.clojure/clojurescript "1.7.48"]

Elapsed time: 22.678 sec (initial compile)
Elapsed time: 6.057 sec (1st incremental compile)
Elapsed time: 4.028 sec (2nd incremental compile)
Elapsed time: 4.370 sec
Elapsed time: 4.105 sec

So the dog is buried somewhere here:

~/github.com/adzerk-oss/boot-cljs> git diff 0.0-3308-0 1.7.48-0

Do I understand well there is no limit on the number of concurrent CLJS compilers running here? : https://github.com/adzerk-oss/boot-cljs/blob/master/src/adzerk/boot_cljs.clj#L218

Though I only have 1 index.html.cljs.edn file in the above mentioned project, so parallel compilation contention can't be the issue...

martinklepsch commented 9 years ago

One thing that always should be done when performance issues arise is passing :compiler-options {:verbose true} to the compiler. Through macros and other stuff there are few limits what can be done during compile time and sometimes these steps can take some time.

Using :verbose true it's easy to identify namespaces that take unusual amounts of time to compile.

Deraen commented 8 years ago

Has anyone ran benchmarks lately? I believe that since 1.7.170 we should be on par with Cljsbuild except for some latency caused by filesets.

arichiardi commented 7 years ago

I have had quite a random feeling that figwheel does things faster (at least according to the "Elapsed Time: ..." string). I didn't have time for rigorous testing though.

martinklepsch commented 7 years ago

Did you observe that difference with a version of boot-figwheel that uses the fileset? On Sun, 20 Nov 2016 at 11:36, Andrea Richiardi notifications@github.com wrote:

I have had quite a random feeling that figwheel does things faster (at least accordong to the "Elapsed Time: ..." string. I didn't have time for rigorous testing though.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/adzerk-oss/boot-cljs/issues/93#issuecomment-261758579, or mute the thread https://github.com/notifications/unsubscribe-auth/AAF82ILO_9blUftOMyGy5Mdw8nWt_znSks5q_85ggaJpZM4F0uFO .

arichiardi commented 7 years ago

Mmm nope. .my version with boot-figheel with proper fileset support has been abandoned..too hackish and was basically forcing sync steps in order to make the fileset mutable. I am now trying to embed the figwheel client in boot-reload. But that in theory shouldn't touch compilation performance...

It would be great to extract and compare only the compilation step with and without fileset. -ar

vikeri commented 7 years ago

I can confirm that there is still significant performance differences between boot-cljs and lein cljs-build, not on the startup (if one cleans the build using lein clean before), but on the recompilation.

startup

tool	time
lein cljsbuild (without lein clean)	2.4 sec
lein cljsbuild (with lein clean)	24.6 sec
boot-cljs	19.9 sec

recompilation (avg)

tool	time
lein cljsbuild	0.8 sec
boot-cljs	5 sec

I'm on a MacBookPro mid-2014

[adzerk/boot-cljs "1.7.228-2" :scope "test"]
[org.clojure/clojurescript "1.9.494"]
[lein-cljsbuild "1.1.5"]

  {:recompile-dependents false
   :optimizations        :none
   :verbose              false
   :cache-analysis       true
   :parallel-build       true
   :target               :nodejs}

martinklepsch commented 7 years ago

@vikeri have you tried running with :verbose and comparing the outputs? If you could put those in a gist that might help.

Generally posting stats here is good, it confirms that the problem still exists, but it unfortunately does not help in figuring out why this is happening. Anything that's public/reproducible etc. would go a long way in getting to the ground of this.

Deraen commented 7 years ago

This is often referred as "cause" when there is performance problems with Boot-cljs. But, to my knowledge there are no bugs that cause considerable difference between Boot-cljs and other tooling. Boot filesets/tmp-dirs casue some overhead, maybe 20%, but now much.

I'm closing this to reduce confusion. If there are cases where Boot-cljs is something like two times slower, new issue should be opened, but I'm afraid I can't do much without project where I can reproduce this.

boot-clj / boot-cljs

Performance Improvements #93

Optimizations :none

Optimizations :advanced

Incremental compiles

Cold build once

Cold build + incremental changes

Advanced build

Cold build once

Cold build + incremental changes

Advanced build

Table of Contents

cloc (not really informing imho)

boot-cljs inital with cljs.edn

boot-cljs incremental with cljs.edn

time lein cljsbuild once dev

lein cljsbuild auto dev

startup

recompilation (avg)