Klebert-Engineering / simfil

The Simple Map Filter Language Interpreter 🌳.
BSD 3-Clause "New" or "Revised" License
6 stars 0 forks source link

Performance Tests [post arena] #30

Closed johannes-wolf closed 1 year ago

github-actions[bot] commented 1 year ago

Test Results

βŸβ€„1 files  Β±0β€‚β€ƒβŸβ€„1 suites  Β±0   15s :stopwatch: +12s 35 tests +1  35 :heavy_check_mark: +1  0 :zzz: Β±0  0 :x: Β±0  36 runs  +1  36 :heavy_check_mark: +1  0 :zzz: Β±0  0 :x: Β±0 

Results for commit 35d92fe5. ± Comparison against base commit cd9e390c.

:recycle: This comment has been updated with latest results.

johannes-wolf commented 1 year ago

Pre #19 Results (GitHub CI) (pr #28)

25: benchmark name                       samples       iterations    estimated
25:                                      mean          low mean      high mean
25:                                      std dev       low std dev   high std dev
25: -------------------------------------------------------------------------------
25: Query typeof Recursive                         100             1     3.44935 s 
25:                                         34.4949 ms    34.4846 ms     34.508 ms 
25:                                           59.02 us    47.1425 us    75.8362 us 
25:                                                                                
25: Query field id Recursive                       100             1     2.09223 s 
25:                                         20.4813 ms    20.4315 ms     20.537 ms 
25:                                         268.792 us     245.06 us    286.361 us 
25:                                                                                
25: Query field id                                 100             1    32.6139 ms 
25:                                          324.59 us    324.031 us    325.933 us 
25:                                         4.17649 us    2.19362 us    8.46351 us 

Post #19 Results (GitHub CI) (this pr)

With area rw-lock

Show (7.04744 s, 5.66519 s, 69.7151 ms) ``` 32: benchmark name samples iterations estimated 32: mean low mean high mean 32: std dev low std dev high std dev 32: ------------------------------------------------------------------------------- 32: Query typeof Recursive 100 1 7.04744 s 32: 70.4019 ms 70.3893 ms 70.4191 ms 32: 74.515 us 59.3381 us 109.083 us 32: 32: Query field id Recursive 100 1 5.66519 s 32: 56.6764 ms 56.6344 ms 56.8626 ms 32: 389.166 us 51.4331 us 921.133 us 32: 32: Query field id 100 1 69.7151 ms 32: 688.586 us 686.385 us 694.812 us 32: 17.3414 us 7.22921 us 36.7479 us ```

Without arena rw-lock

32: benchmark name                       samples       iterations    estimated
32:                                      mean          low mean      high mean
32:                                      std dev       low std dev   high std dev
32: -------------------------------------------------------------------------------
32: Query typeof Recursive                         100             1     7.27451 s 
32:                                         73.4406 ms    73.1747 ms    73.9618 ms 
32:                                         1.82248 ms    1.05426 ms    2.84808 ms 
32:                                                                                
32: Query field id Recursive                       100             1     6.03972 s 
32:                                         60.1127 ms    60.0544 ms    60.1633 ms 
32:                                         275.516 us     238.19 us    327.208 us 
32:                                                                                
32: Query field id                                 100             1    71.4648 ms 
32:                                         714.482 us    712.938 us    716.781 us 
32:                                         9.47387 us    6.78269 us    13.0234 us 

With removal of std::function from this pr

32: benchmark name                       samples       iterations    estimated
32:                                      mean          low mean      high mean
32:                                      std dev       low std dev   high std dev
32: -------------------------------------------------------------------------------
32: Query typeof Recursive                         100             1     5.86631 s 
32:                                         58.6589 ms    58.6156 ms    58.7192 ms 
32:                                         257.943 us    186.701 us    416.353 us 
32:                                                                                
32: Query field id Recursive                       100             1     4.94557 s 
32:                                         50.0428 ms    49.9219 ms    50.1675 ms 
32:                                         627.646 us    576.839 us     684.89 us 
32:                                                                                
32: Query field id                                 100             1    57.9909 ms 
32:                                         564.763 us    562.932 us    567.046 us 
32:                                          10.398 us     8.6934 us     12.199 us 
github-actions[bot] commented 1 year ago
Package Line Rate Branch Rate Health
include.simfil 54% 34% βž–
include.simfil.model 77% 39% βž–
src 63% 35% βž–
src.model 70% 40% βž–
Summary 63% (2062 / 3267) 35% (1728 / 4879) βž–

Minimum allowed line rate is 50%