weavejester
commented
1 year ago Thanks for the PR. Those are some very helpful optimizations. To answer your questions:
Why are we sorting the attributes and the style attribute? Is this just for test output?
It's sorted to produce a consistent output, so a particular input will always produce the same output. If this weren't the case, the attribute order could change, as map keys are unordered.
Why do we escaping HTML text/attributes by default? Isn't this a complex operation?
For safety. The consequences of client data being unescaped is significant enough that it's worth erring on the side of security over speed in this case.
Panthevm
Introduction
When I set out to measure the number of requests per second from my server—specifically, those outputting a small HTML string based on hiccup — I was surprised.
Despite the hiccup content being very small, the response using the hiccup->html conversion was discernibly slower than the response with the identical HTML string, showing a disparity of 24k req/sec versus 40k req/sec.
Upon delving into the hiccup source code, I identified several factors that significantly hampered the processing speed.
Benchmarking & Profiling
Initially, I decided to estimate the speed of execution with criterium. On my computer, I obtained the following results:
Case №1
``` clj (def hiccup [:span "hello"]) (criterium.core/bench (hiccup2.core/html hiccup)) ;; Evaluation count : 25789500 in 60 samples of 429825 calls. ;; Execution time mean : 2.330993 µs ;; Execution time std-deviation : 41.090772 ns ;; Execution time lower quantile : 2.303320 µs ( 2.5%) ;; Execution time upper quantile : 2.373499 µs (97.5%) ;; Overhead used : 7.614148 ns ;; Found 6 outliers in 60 samples (10.0000 %) ;; low-severe 1 (1.6667 %) ;; low-mild 4 (6.6667 %) ;; high-mild 1 (1.6667 %) ;; Variance from outliers : 6.2942 % Variance is slightly inflated by outliers ```Case №2
``` clj (def hiccup [:span#foo.bar.baz {:id "foo" :class "qux" :styles {:a 1 :b 2}} "hello"]) (criterium.core/bench (hiccup2.core/html hiccup)) ;; Evaluation count : 8239680 in 60 samples of 137328 calls. ;; Execution time mean : 7.269213 µs ;; Execution time std-deviation : 145.634319 ns ;; Execution time lower quantile : 7.167641 µs ( 2.5%) ;; Execution time upper quantile : 7.541402 µs (97.5%) ;; Overhead used : 7.614148 ns ;; Found 5 outliers in 60 samples (8.3333 %) ;; low-severe 2 (3.3333 %) ;; low-mild 3 (5.0000 %) ;; Variance from outliers : 9.3512 % Variance is slightly inflated by outliers ```Case №3
``` clj (def hiccup [:html [:head [:title "Title"]] [:body [:span "foo"] [:span#id "foo"] [:span#id.class1.class2 "foo"] [:span#id.class1.class2 {:id "id"} "foo"] [:span#id.class1.class2 {:class "class3"} "foo"] [:span#id.class1.class2 {:styles {:a 1 :b 2}} "foo"] (for [i [1 2 3 4]] [:span i])]]) (criterium.core/bench (hiccup2.core/html hiccup)) ;; Evaluation count : 1288140 in 60 samples of 21469 calls. ;; Execution time mean : 46.760425 µs ;; Execution time std-deviation : 837.581889 ns ;; Execution time lower quantile : 46.220047 µs ( 2.5%) ;; Execution time upper quantile : 47.416694 µs (97.5%) ;; Overhead used : 7.614148 ns ;; Found 2 outliers in 60 samples (3.3333 %) ;; low-severe 1 (1.6667 %) ;; low-mild 1 (1.6667 %) ;; Variance from outliers : 7.7697 % Variance is slightly inflated by outliers ```After that I decided to do the profiling using clj-async-profiler. I discovered numerous calls where there was a cast to
clojure.lang.RT.LazySeq.seqand a call toclojure.lang.RestFn.invoke. These calls are highlighted in dark pink in the picture.Profiling report
Optimization Changes
->hiccup.util/to-str [x]->hiccup.util/build-string [& strs] (macro)->StringBuilder + hiccup.util/fast-run->hiccup.compiler/parse-tag->hiccup.compiler/parse-tagResults
Case №1 (2.4µs -> 600ns)
``` clj (def hiccup [:span "hello"]) (criterium.core/bench (hiccup2.core/html hiccup)) ;; Evaluation count : 100645620 in 60 samples of 1677427 calls. ;; Execution time mean : 598.744611 ns ;; Execution time std-deviation : 11.053241 ns ;; Execution time lower quantile : 566.805045 ns ( 2.5%) ;; Execution time upper quantile : 614.233694 ns (97.5%) ;; Overhead used : 8.220649 ns ;; Found 8 outliers in 60 samples (13.3333 %) ;; low-severe 4 (6.6667 %) ;; low-mild 2 (3.3333 %) ;; high-mild 2 (3.3333 %) ;; Variance from outliers : 7.8027 % Variance is slightly inflated by outliers ```Case №2 (7.3 µs -> 2.9 µs)
``` clj (def hiccup [:span#foo.bar.baz {:id "foo" :class "qux" :styles {:a 1 :b 2}} "hello"]) (criterium.core/bench (hiccup2.core/html hiccup)) ;; Evaluation count : 20125980 in 60 samples of 335433 calls. ;; Execution time mean : 2.942334 µs ;; Execution time std-deviation : 26.560566 ns ;; Execution time lower quantile : 2.911728 µs ( 2.5%) ;; Execution time upper quantile : 3.007485 µs (97.5%) ;; Overhead used : 8.220649 ns ;; Found 5 outliers in 60 samples (8.3333 %) ;; low-severe 5 (8.3333 %) ;; Variance from outliers : 1.6389 % Variance is slightly inflated by outliers ```Case №3 (46.8µs -> 18 µs)
``` clj (def hiccup [:html [:head [:title "Title"]] [:body [:span "foo"] [:span#id "foo"] [:span#id.class1.class2 "foo"] [:span#id.class1.class2 {:id "id"} "foo"] [:span#id.class1.class2 {:class "class3"} "foo"] [:span#id.class1.class2 {:styles {:a 1 :b 2}} "foo"] (for [i [1 2 3 4]] [:span i])]]) (criterium.core/bench (hiccup2.core/html hiccup)) ;; Evaluation count : 3370440 in 60 samples of 56174 calls. ;; Execution time mean : 18.236171 µs ;; Execution time std-deviation : 383.229484 ns ;; Execution time lower quantile : 17.635168 µs ( 2.5%) ;; Execution time upper quantile : 18.784032 µs (97.5%) ;; Overhead used : 8.220649 ns ;; Found 1 outliers in 60 samples (1.6667 %) ;; low-severe 1 (1.6667 %) ;; Variance from outliers : 9.4103 % Variance is slightly inflated by outliers ```Profiling report
Questions