turion / rhine

Haskell Functional Reactive Programming framework with type-level clocks
http://hackage.haskell.org/package/rhine
122 stars 21 forks source link

Add benchmarks #285

Closed turion closed 6 months ago

turion commented 8 months ago
turion commented 6 months ago

Possibly cherry pick https://github.com/turion/rhine/pull/299/commits/adb464a8044e27454beb91555a0a7746da38cad2?

turion commented 6 months ago

Is it legal to carry the Gutenberg file around?

I found another version of the file that is certainly legal to use here.

turion commented 6 months ago

The new benchmark data file has more lines, which makes it even slower:

benchmarking WordCount/rhine
time                 3.733 s    (3.474 s .. 3.992 s)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 3.738 s    (3.706 s .. 3.783 s)
std dev              43.07 ms   (15.49 ms .. 56.79 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking WordCount/dunai
time                 999.1 ms   (912.9 ms .. 1.076 s)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 1.032 s    (1.013 s .. 1.046 s)
std dev              19.56 ms   (9.140 ms .. 27.15 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking WordCount/Text/IORef
time                 81.32 ms   (80.70 ms .. 82.68 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 80.79 ms   (80.39 ms .. 81.25 ms)
std dev              745.6 μs   (506.6 μs .. 1.029 ms)

benchmarking WordCount/Text/no IORef
time                 164.9 ms   (130.1 ms .. 213.0 ms)
                     0.952 R²   (0.847 R² .. 0.999 R²)
mean                 173.6 ms   (162.1 ms .. 188.7 ms)
std dev              18.36 ms   (12.95 ms .. 24.35 ms)
variance introduced by outliers: 27% (moderately inflated)

benchmarking WordCount/Text/Lazy
time                 100.6 ms   (92.54 ms .. 108.5 ms)
                     0.991 R²   (0.981 R² .. 1.000 R²)
mean                 100.6 ms   (98.10 ms .. 103.2 ms)
std dev              4.322 ms   (3.407 ms .. 6.058 ms)

It's clear that Rhine and dunai have an overhead per tick, since the benchmark programs operate line-wise.