search

kkebo / zyphy

Zyphy is (or will be) a fast web browser engine written in Swift.
MIT License
4 stars 2 forks source link

feat(benchmark)!: replace the benchmark data with html5ever's #59

Closed kkebo closed 5 months ago

kkebo commented 5 months ago

fixes #17

These data are not so big, but we can compare performance with html5ever now.

Benchmark results

M1 Mac mini, Fedora 39 aarch64 (Asahi Linux)

zyphy:

$ swift --version
Swift version 6.0-dev (LLVM ef51a35f858944e, Swift f36098a14a0d48b)
Target: aarch64-unknown-linux-gnu
$ swift package --package-path Benchmarks benchmark

==================
Running Benchmarks
==================

100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmark:lipsum
100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmark:lipsum-zh
100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmark:medium-fragment
100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmark:small-fragment
100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmark:strong
100% [------------------------------------------------------------] ETA: 00:00:00 | MyBenchmark:tiny-fragment

===================================================
Baseline 'Current_run'
===================================================

Host 'Brown-rhinoceros-beetle' with 8 'aarch64' processors with 7 GB memory, running:
#1 SMP PREEMPT_DYNAMIC Sun Mar 24 19:44:17 UTC 2024

===========
MyBenchmark
===========

lipsum
╒══════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                   │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞══════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Time (wall clock) (μs) * │     775 │     775 │     776 │     782 │     783 │     786 │     786 │      78 │
╘══════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

lipsum-zh
╒══════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                   │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞══════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Time (wall clock) (μs) * │     153 │     153 │     153 │     154 │     155 │     159 │     160 │     100 │
╘══════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

medium-fragment
╒══════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                   │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞══════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Time (wall clock) (μs) * │     357 │     357 │     357 │     358 │     364 │     365 │     368 │     100 │
╘══════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

small-fragment
╒══════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                   │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞══════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Time (wall clock) (μs) * │      58 │      58 │      58 │      58 │      58 │      62 │      62 │     100 │
╘══════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

strong
╒══════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                   │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞══════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Time (wall clock) (μs) * │     100 │     100 │     100 │     100 │     101 │     106 │     106 │     100 │
╘══════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

tiny-fragment
╒══════════════════════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╤═════════╕
│ Metric                   │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞══════════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Time (wall clock) (ns) * │    1599 │    1602 │    1605 │    1715 │    1814 │    1979 │    1980 │     100 │
╘══════════════════════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╧═════════╛

html5ever (1ae2de3a1796a9b52a804a02039c6c1499e2f461):

$ cargo --version
cargo 1.77.2 (e52e36006 2024-03-26)
$ cargo bench
    Finished bench [optimized] target(s) in 0.02s                                                                                                                                       
     Running unittests src/lib.rs (/home/kebo/html5ever/target/release/deps/html5ever-7830d7536019e374)                                                                                 

running 9 tests                                                                                                                                                                         
test tokenizer::test::check_lines ... ignored                                                                                                                                           
test tokenizer::test::check_lines_with_new_line ... ignored                                                                                                                             
test tokenizer::test::push_to_None_gives_singleton ... ignored                                                                                                                          
test tokenizer::test::push_to_empty_appends ... ignored                                                                                                                                 
test tokenizer::test::push_to_nonempty_appends ... ignored                                                                                                                              
test util::str::test::lower_letter_A_is_a ... ignored                                                                                                                                   
test util::str::test::lower_letter_a_is_a ... ignored                                                                                                                                   
test util::str::test::lower_letter_nonascii_is_None ... ignored                                                                                                                         
test util::str::test::lower_letter_symbol_is_None ... ignored                                                                                                                           

test result: ok. 0 passed; 0 failed; 9 ignored; 0 measured; 0 filtered out; finished in 0.00s                                                                                           

     Running benches/html5ever.rs (/home/kebo/html5ever/target/release/deps/html5ever-2692e4c685e13aa1)                                                                                 
Gnuplot not found, using plotters backend                                                                                                                                               
html tokenizing lipsum.html                                                                                                                                                             
                        time:   [14.285 µs 14.330 µs 14.387 µs]                                                                                                                         
                        change: [-0.4645% -0.0463% +0.3596%] (p = 0.83 > 0.05)                                                                                                          
                        No change in performance detected.                                                                                                                              
Found 15 outliers among 100 measurements (15.00%)                                                                                                                                       
  3 (3.00%) high mild                                                                                                                                                                   
  12 (12.00%) high severe                                                                                                                                                               

html tokenizing lipsum-zh.html                                                                                                                                                          
                        time:   [7.3118 µs 7.3312 µs 7.3563 µs]                                                                                                                         
                        change: [-0.2412% +0.1385% +0.5216%] (p = 0.49 > 0.05)                                                                                                          
                        No change in performance detected.                                                                                                                              
Found 11 outliers among 100 measurements (11.00%)                                                                                                                                       
  1 (1.00%) high mild                                                                                                                                                                   
  10 (10.00%) high severe                                                                                                                                                               

html tokenizing medium-fragment.html
                        time:   [31.061 µs 31.141 µs 31.250 µs]
                        change: [-0.3174% +0.0544% +0.4588%] (p = 0.77 > 0.05)
                        No change in performance detected.
Found 18 outliers among 100 measurements (18.00%)
  6 (6.00%) high mild
  12 (12.00%) high severe

html tokenizing small-fragment.html
                        time:   [3.0237 µs 3.0337 µs 3.0455 µs]
                        change: [+0.3064% +0.6608% +1.0970%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
  8 (8.00%) high mild
  5 (5.00%) high severe

html tokenizing tiny-fragment.html
                        time:   [299.64 ns 300.56 ns 301.71 ns]
                        change: [-0.6318% -0.2531% +0.1055%] (p = 0.19 > 0.05)
                        No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
  5 (5.00%) high mild
  6 (6.00%) high severe

html tokenizing strong.html
                        time:   [18.107 µs 18.179 µs 18.269 µs]
                        change: [+0.0617% +0.6264% +1.1502%] (p = 0.03 < 0.05)              
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high mild
github-actions[bot] commented 5 months ago

Code Metrics Report

main (9fe7e04) #59 (31d826e) +/-
Coverage 88.7% 88.7% 0.0%
Details ``` diff | | main (9fe7e04) | #59 (31d826e) | +/- | |-----------|----------------|---------------|------| | Coverage | 88.7% | 88.7% | 0.0% | | Files | 8 | 8 | 0 | | Lines | 1258 | 1258 | 0 | | Covered | 1116 | 1116 | 0 | ```

Reported by octocov

github-actions[bot] commented 5 months ago

Code Metrics Report

main (9fe7e04) #59 (27fbba8) +/-
Coverage 88.7% 88.7% 0.0%
Details ``` diff | | main (9fe7e04) | #59 (27fbba8) | +/- | |-----------|----------------|---------------|------| | Coverage | 88.7% | 88.7% | 0.0% | | Files | 8 | 8 | 0 | | Lines | 1258 | 1258 | 0 | | Covered | 1116 | 1116 | 0 | ```

Reported by octocov

github-actions[bot] commented 5 months ago

Code Metrics Report

main (1379f9a) #59 (d93f40a) +/-
Coverage 88.7% 88.7% 0.0%
Details ``` diff | | main (1379f9a) | #59 (d93f40a) | +/- | |-----------|----------------|---------------|------| | Coverage | 88.7% | 88.7% | 0.0% | | Files | 8 | 8 | 0 | | Lines | 1258 | 1258 | 0 | | Covered | 1116 | 1116 | 0 | ```

Reported by octocov

github-actions[bot] commented 5 months ago

Code Metrics Report

main (1379f9a) #59 (99148d3) +/-
Coverage 88.7% 88.7% 0.0%
Details ``` diff | | main (1379f9a) | #59 (99148d3) | +/- | |-----------|----------------|---------------|------| | Coverage | 88.7% | 88.7% | 0.0% | | Files | 8 | 8 | 0 | | Lines | 1258 | 1258 | 0 | | Covered | 1116 | 1116 | 0 | ```

Reported by octocov

github-actions[bot] commented 5 months ago

Code Metrics Report

main (1379f9a) #59 (ed246eb) +/-
Coverage 88.7% 88.7% 0.0%
Details ``` diff | | main (1379f9a) | #59 (ed246eb) | +/- | |-----------|----------------|---------------|------| | Coverage | 88.7% | 88.7% | 0.0% | | Files | 8 | 8 | 0 | | Lines | 1258 | 1258 | 0 | | Covered | 1116 | 1116 | 0 | ```

Reported by octocov