search

anmonteiro / lumo

Fast, cross-platform, standalone ClojureScript environment
Eclipse Public License 1.0
1.88k stars 84 forks source link

Slower compared to planck interpreting a file #13

Closed tiye closed 7 years ago

tiye commented 7 years ago

Lumo turned out to be slower in my project:

=>> time lumo -c $boot_deps:src/ -i render.cljs

real    0m23.149s
user    0m21.812s
sys 0m1.913s

=>> time planck -c $boot_deps:src/ -i render.cljs

real    0m5.097s
user    0m7.212s
sys 0m0.301s

I'm using a personal workflow that pre-renders HTML with cljs. The script is render.cljs.

export boot_deps=`boot show -c`
lumo -c $boot_deps:src/ -i render.cljs

In order to render that, I have to load some dependencies:

(ns ssr-stages.boot
  (:require
    [respo.alias :refer [html head title script style meta' div link body]]
    [respo.render.html :refer [make-html make-string]]
    [stack-workflow.comp.container :refer [comp-container]]
    [planck.core :refer [spit]]))

from quite some classpathes:

=>> echo $boot_deps
/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/System/Library/Java/Extensions/MRJToolkit.jar:/usr/local/Cellar/boot-clj/2.5.2/bin/boot:/Users/chen/.boot/cache/bin/2.6.0/boot.jar:/Users/chen/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar:/Users/chen/.m2/repository/boot/core/2.6.0/core-2.6.0.jar:/Users/chen/.m2/repository/boot/pod/2.6.0/pod-2.6.0.jar:/Users/chen/.m2/repository/org/tcrawley/dynapath/0.2.3/dynapath-0.2.3.jar:/Users/chen/.m2/repository/org/projectodd/shimdandy/shimdandy-impl/1.2.0/shimdandy-impl-1.2.0.jar:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1nye/11lgcg/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1nye/6qp7pj/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1nye/rcesf6/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1nye/-x24pa9/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1nye/-lsqdbp/:/Users/chen/.m2/repository/respo/respo/0.3.31/respo-0.3.31.jar:/Users/chen/.m2/repository/org/clojure/clojurescript/1.9.293/clojurescript-1.9.293.jar:/Users/chen/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20160911/closure-compiler-unshaded-v20160911.jar:/Users/chen/.m2/repository/com/google/jsinterop/jsinterop-annotations/1.0.0/jsinterop-annotations-1.0.0.jar:/Users/chen/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/chen/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/chen/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/chen/.m2/repository/org/clojure/data.json/0.2.6/data.json-0.2.6.jar:/Users/chen/.m2/repository/adzerk/boot-test/1.1.2/boot-test-1.1.2.jar:/Users/chen/.m2/repository/org/mozilla/rhino/1.7R5/rhino-1.7R5.jar:/Users/chen/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/chen/.m2/repository/com/google/javascript/closure-compiler-externs/v20160911/closure-compiler-externs-v20160911.jar:/Users/chen/.m2/repository/adzerk/boot-reload/0.4.12/boot-reload-0.4.12.jar:/Users/chen/.m2/repository/mvc-works/hsl/0.1.2/hsl-0.1.2.jar:/Users/chen/.m2/repository/respo/ui/0.1.4/ui-0.1.4.jar:/Users/chen/.m2/repository/org/clojure/tools.reader/1.0.0-beta3/tools.reader-1.0.0-beta3.jar:/Users/chen/.m2/repository/args4j/args4j/2.0.26/args4j-2.0.26.jar:/Users/chen/.m2/repository/org/clojure/google-closure-library/0.0-20160609-f42b4a24/google-closure-library-0.0-20160609-f42b4a24.jar:/Users/chen/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20160609-f42b4a24/google-closure-library-third-party-0.0-20160609-f42b4a24.jar:/Users/chen/.m2/repository/cirru/boot-stack-server/0.1.22/boot-stack-server-0.1.22.jar:/Users/chen/.m2/repository/clansi/clansi/1.0.0/clansi-1.0.0.jar:/Users/chen/.m2/repository/ring-cors/ring-cors/0.1.8/ring-cors-0.1.8.jar:/Users/chen/.m2/repository/cirru/sepal/0.0.14/sepal-0.0.14.jar:/Users/chen/.m2/repository/cumulo/shallow-diff/0.1.1/shallow-diff-0.1.1.jar:/Users/chen/.m2/repository/cirru/parser/0.0.3/parser-0.0.3.jar:/Users/chen/.m2/repository/fipp/fipp/0.6.6/fipp-0.6.6.jar:/Users/chen/.m2/repository/org/clojure/core.rrb-vector/0.0.11/core.rrb-vector-0.0.11.jar:/Users/chen/.m2/repository/cheshire/cheshire/5.5.0/cheshire-5.5.0.jar:/Users/chen/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.5.3/jackson-dataformat-smile-2.5.3.jar:/Users/chen/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar:/Users/chen/.m2/repository/tigris/tigris/0.1.1/tigris-0.1.1.jar:/Users/chen/.m2/repository/ring/ring-jetty-adapter/1.5.0/ring-jetty-adapter-1.5.0.jar:/Users/chen/.m2/repository/ring/ring-servlet/1.5.0/ring-servlet-1.5.0.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-server/9.2.10.v20150310/jetty-server-9.2.10.v20150310.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-io/9.2.10.v20150310/jetty-io-9.2.10.v20150310.jar:/Users/chen/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/chen/.m2/repository/ring/ring-core/1.5.0/ring-core-1.5.0.jar:/Users/chen/.m2/repository/clj-time/clj-time/0.11.0/clj-time-0.11.0.jar:/Users/chen/.m2/repository/joda-time/joda-time/2.8.2/joda-time-2.8.2.jar:/Users/chen/.m2/repository/crypto-equality/crypto-equality/1.0.0/crypto-equality-1.0.0.jar:/Users/chen/.m2/repository/commons-io/commons-io/2.5/commons-io-2.5.jar:/Users/chen/.m2/repository/ring/ring-codec/1.0.1/ring-codec-1.0.1.jar:/Users/chen/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar:/Users/chen/.m2/repository/commons-fileupload/commons-fileupload/1.3.1/commons-fileupload-1.3.1.jar:/Users/chen/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-http/9.2.10.v20150310/jetty-http-9.2.10.v20150310.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-util/9.2.10.v20150310/jetty-util-9.2.10.v20150310.jar:/Users/chen/.m2/repository/adzerk/boot-cljs/1.7.228-1/boot-cljs-1.7.228-1.jar:/Users/chen/.m2/repository/crypto-random/crypto-random/1.2.0/crypto-random-1.2.0.jar

So why it's slow?

anmonteiro commented 7 years ago

Planck runs on JavaScriptCore, Lumo runs on Node.js/V8.

When David Nolen was optimizing the ClojureScript compiler for self-hosted ClojureScript, he found out that Node.js was around 2.5x slower than JavaScriptCore.

I suppose there is nothing that we can do about this, unless further optimizations upstream in the ClojureScript compiler (if possible).

On another note, if you're not already doing that, you can pass the -K or -k path option to Lumo so that it caches the result of loading dependencies. This way, the first run will actually compile your dependencies, but subsequent executions should be able to utilize the cache output, making it run faster.

tiye commented 7 years ago

Turned out my code make caching broken:

=>> lumo -k -c $boot_deps:src/ -i render.cljs
#error {:message ERROR in file respo/util/detect.cljc, :data {:tag :cljs/analysis-error}, :cause #error {:message Failed writing cache to /respo_SLASH_util_SLASH_detect.js, :data {:js-error #object[Error Error: EACCES: permission denied, open '/respo_SLASH_util_SLASH_detect.js']}}}
anmonteiro commented 7 years ago

you need to pass a path to -k. e.g. lumo -k cache-dir ...

anmonteiro commented 7 years ago

That's still a cryptic error and we should improve the error message in that case.

tiye commented 7 years ago

Is there a default cache dir? I think the error itself is cached:

=>> lumo -k caches/ $boot_deps:src/ -i render.cljs
#error {:message Could not load file /Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/jaccess.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/System/Library/Java/Extensions/MRJToolkit.jar:/usr/local/Cellar/boot-clj/2.5.2/bin/boot:/Users/chen/.boot/cache/bin/2.6.0/boot.jar:/Users/chen/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar:/Users/chen/.m2/repository/boot/core/2.6.0/core-2.6.0.jar:/Users/chen/.m2/repository/boot/pod/2.6.0/pod-2.6.0.jar:/Users/chen/.m2/repository/org/tcrawley/dynapath/0.2.3/dynapath-0.2.3.jar:/Users/chen/.m2/repository/org/projectodd/shimdandy/shimdandy-impl/1.2.0/shimdandy-impl-1.2.0.jar:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1ou4/11lgcg/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1ou4/6qp7pj/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1ou4/rcesf6/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1ou4/-x24pa9/:/Users/chen/.boot/cache/tmp/Users/chen/repo/mvc-works/stack-workflow/1ou4/-lsqdbp/:/Users/chen/.m2/repository/respo/respo/0.3.31/respo-0.3.31.jar:/Users/chen/.m2/repository/org/clojure/clojurescript/1.9.293/clojurescript-1.9.293.jar:/Users/chen/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20160911/closure-compiler-unshaded-v20160911.jar:/Users/chen/.m2/repository/com/google/jsinterop/jsinterop-annotations/1.0.0/jsinterop-annotations-1.0.0.jar:/Users/chen/.m2/repository/com/google/code/gson/gson/2.2.4/gson-2.2.4.jar:/Users/chen/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/chen/.m2/repository/com/google/guava/guava/19.0/guava-19.0.jar:/Users/chen/.m2/repository/org/clojure/data.json/0.2.6/data.json-0.2.6.jar:/Users/chen/.m2/repository/adzerk/boot-test/1.1.2/boot-test-1.1.2.jar:/Users/chen/.m2/repository/org/mozilla/rhino/1.7R5/rhino-1.7R5.jar:/Users/chen/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/chen/.m2/repository/com/google/javascript/closure-compiler-externs/v20160911/closure-compiler-externs-v20160911.jar:/Users/chen/.m2/repository/adzerk/boot-reload/0.4.12/boot-reload-0.4.12.jar:/Users/chen/.m2/repository/mvc-works/hsl/0.1.2/hsl-0.1.2.jar:/Users/chen/.m2/repository/respo/ui/0.1.4/ui-0.1.4.jar:/Users/chen/.m2/repository/org/clojure/tools.reader/1.0.0-beta3/tools.reader-1.0.0-beta3.jar:/Users/chen/.m2/repository/args4j/args4j/2.0.26/args4j-2.0.26.jar:/Users/chen/.m2/repository/org/clojure/google-closure-library/0.0-20160609-f42b4a24/google-closure-library-0.0-20160609-f42b4a24.jar:/Users/chen/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20160609-f42b4a24/google-closure-library-third-party-0.0-20160609-f42b4a24.jar:/Users/chen/.m2/repository/cirru/boot-stack-server/0.1.22/boot-stack-server-0.1.22.jar:/Users/chen/.m2/repository/clansi/clansi/1.0.0/clansi-1.0.0.jar:/Users/chen/.m2/repository/ring-cors/ring-cors/0.1.8/ring-cors-0.1.8.jar:/Users/chen/.m2/repository/cirru/sepal/0.0.14/sepal-0.0.14.jar:/Users/chen/.m2/repository/cumulo/shallow-diff/0.1.1/shallow-diff-0.1.1.jar:/Users/chen/.m2/repository/cirru/parser/0.0.3/parser-0.0.3.jar:/Users/chen/.m2/repository/fipp/fipp/0.6.6/fipp-0.6.6.jar:/Users/chen/.m2/repository/org/clojure/core.rrb-vector/0.0.11/core.rrb-vector-0.0.11.jar:/Users/chen/.m2/repository/cheshire/cheshire/5.5.0/cheshire-5.5.0.jar:/Users/chen/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-smile/2.5.3/jackson-dataformat-smile-2.5.3.jar:/Users/chen/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar:/Users/chen/.m2/repository/tigris/tigris/0.1.1/tigris-0.1.1.jar:/Users/chen/.m2/repository/ring/ring-jetty-adapter/1.5.0/ring-jetty-adapter-1.5.0.jar:/Users/chen/.m2/repository/ring/ring-servlet/1.5.0/ring-servlet-1.5.0.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-server/9.2.10.v20150310/jetty-server-9.2.10.v20150310.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-io/9.2.10.v20150310/jetty-io-9.2.10.v20150310.jar:/Users/chen/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:/Users/chen/.m2/repository/ring/ring-core/1.5.0/ring-core-1.5.0.jar:/Users/chen/.m2/repository/clj-time/clj-time/0.11.0/clj-time-0.11.0.jar:/Users/chen/.m2/repository/joda-time/joda-time/2.8.2/joda-time-2.8.2.jar:/Users/chen/.m2/repository/crypto-equality/crypto-equality/1.0.0/crypto-equality-1.0.0.jar:/Users/chen/.m2/repository/commons-io/commons-io/2.5/commons-io-2.5.jar:/Users/chen/.m2/repository/ring/ring-codec/1.0.1/ring-codec-1.0.1.jar:/Users/chen/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar:/Users/chen/.m2/repository/commons-fileupload/commons-fileupload/1.3.1/commons-fileupload-1.3.1.jar:/Users/chen/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-http/9.2.10.v20150310/jetty-http-9.2.10.v20150310.jar:/Users/chen/.m2/repository/org/eclipse/jetty/jetty-util/9.2.10.v20150310/jetty-util-9.2.10.v20150310.jar:/Users/chen/.m2/repository/adzerk/boot-cljs/1.7.228-1/boot-cljs-1.7.228-1.jar:/Users/chen/.m2/repository/crypto-random/crypto-random/1.2.0/crypto-random-1.2.0.jar:src/, :data {}}
anmonteiro commented 7 years ago

Opened https://github.com/anmonteiro/lumo/issues/14 to track that

anmonteiro commented 7 years ago

@jiyinyiyong there's only a default cache dir if you pass -K (upper case)

tiye commented 7 years ago

Can I clear the caches manually in this case?

anmonteiro commented 7 years ago

Just delete the folder :)

anmonteiro commented 7 years ago

The default cache path is .lumo_cache at the directory you're running Lumo

tiye commented 7 years ago

I just deleted .lumo_cache/ in my project. The error remains, probably it's just an error not related with cache folder now...

anmonteiro commented 7 years ago

It looks like you're forgetting to also specify the -c option.

This:

lumo -k caches/ $boot_deps:src/ -i render.cljs

should be revised to:

lumo -k caches/  -c $boot_deps:src/ -i render.cljs
tiye commented 7 years ago

My mistake. Now I can compile it without errors. Looks like I can clear caches partially by manually removing some of them. 2.8s now faster than Planck. :)

=>> time lumo -k caches -c $boot_deps:src/ -i render.cljs

real    0m1.761s
user    0m1.893s
sys 0m0.170s
=>> rm caches/stack_workflow_SLASH_comp_SLASH_container.*
=>> time lumo -k caches -c $boot_deps:src/ -i render.cljs

real    0m2.848s
user    0m2.932s
sys 0m0.265s
anmonteiro commented 7 years ago

Awesome, glad it's working. Should we close this then?