Benchmark Results

[mweisgut]

Misusing the issue function here for communicating benchmark results on Nemea. Feel free to close the issue.

Benchmark commits:

• Benchmark robin map: bbe2301e731486a993dcb9b2c15b62fbed65c5c5
• Memory tracking, disabled: 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8
• Memory tracking, activated: f064cb5e9cceb48dded0de01c46ee80924607ec5
• Master: 1c24c5f49719e589324037c5037866a7388a034c

Benchmark results:

• [1] and [2] were moved from the memory tracking PR of the Hyrise repository to this issue. (1st iteration)
• For [3], bbe2301e731486a993dcb9b2c15b62fbed65c5c5 and 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8 were benchmarked again (2nd iteration).
• [4] compares bbe2301e731486a993dcb9b2c15b62fbed65c5c5 and 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8 of the 1st iteration

[mweisgut]

[1] Master vs. RobinMap (Iteration 1)

Command:

../scripts/benchmark_all.sh 1c24c5f49719e589324037c5037866a7388a034c bbe2301e731486a993dcb9b2c15b62fbed65c5c5

System

nemea - click to expand

| property | value | | -- | -- | | Hostname | nemea | | CPU | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | Memory | 1.4TB | | numactl | nodebind: 1 | | numactl | membind: 1 |

Commit Info and Build Time	commit	date	message	build time
1c24c5f49	21.07.2022 21:52	Ensure correct results of EXTRACT expressions (#2482)	real 332.61 user 2798.84 sys 79.02
bbe2301e7	21.07.2022 13:31	Exchanged emplace with try_emplace	real 329.07 user 2770.45 sys 78.28

hyriseBenchmarkTPCH - single-threaded, SF 10.0

Sum of avg. item runtimes: +9% || Geometric mean of throughput changes: -8%

Configuration Overview - click to expand

```diff +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 08:23:46 | 2022-07-24 11:44:41 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 5827.3 | 5791.7 | -1% || 0.17 | 0.17 | +1% | 0.7730 | -| TPC-H 02 || 42.7 | 47.9 | +12%˄ || 23.39 | 20.88 | -11%˄ | 0.0000 | -| TPC-H 03 || 2208.7 | 2849.7 | +29% || 0.45 | 0.35 | -22% | 0.0000 | -| TPC-H 04 || 1399.1 | 1618.7 | +16% || 0.71 | 0.62 | -14% | 0.0000 | -| TPC-H 05 || 3022.5 | 3493.3 | +16% || 0.33 | 0.29 | -13% | 0.0000 | | TPC-H 06 || 156.2 | 153.4 | -2%˄ || 6.40 | 6.52 | +2%˄ | 0.0085 | -| TPC-H 07 || 919.6 | 1053.7 | +15% || 1.09 | 0.95 | -13% | 0.0000 | -| TPC-H 08 || 701.0 | 821.5 | +17% || 1.43 | 1.22 | -15% | 0.0000 | | TPC-H 09 || 5172.5 | 5299.7 | +2% || 0.19 | 0.19 | -2% | 0.0000 | -| TPC-H 10 || 2876.4 | 3167.2 | +10% || 0.35 | 0.32 | -9% | 0.0000 | -| TPC-H 11 || 72.1 | 78.4 | +9%˄ || 13.88 | 12.75 | -8%˄ | 0.0000 | -| TPC-H 12 || 971.4 | 1112.2 | +14% || 1.03 | 0.90 | -13% | 0.0000 | -| TPC-H 13 || 4635.8 | 4943.3 | +7% || 0.22 | 0.20 | -6% | 0.0000 | | TPC-H 14 || 416.4 | 429.4 | +3%˄ || 2.40 | 2.33 | -3%˄ | 0.0000 | | TPC-H 15 || 188.5 | 188.6 | +0%˄ || 5.31 | 5.30 | -0%˄ | 0.2131 | | TPC-H 16 || 628.2 | 625.1 | -0% || 1.59 | 1.60 | +0% | 0.0070 | | TPC-H 17 || 213.7 | 220.1 | +3%˄ || 4.68 | 4.54 | -3%˄ | 0.0000 | | TPC-H 18 || 1517.2 | 1581.7 | +4% || 0.66 | 0.63 | -4% | 0.0006 | | TPC-H 19 || 266.0 | 270.2 | +2%˄ || 3.76 | 3.70 | -2%˄ | 0.0000 | | TPC-H 20 || 397.3 | 410.2 | +3%˄ || 2.52 | 2.44 | -3%˄ | 0.0000 | -| TPC-H 21 || 4693.7 | 5414.3 | +15% || 0.21 | 0.18 | -13% | 0.0000 | -| TPC-H 22 || 422.4 | 476.5 | +13%˄ || 2.37 | 2.10 | -11%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 36748.7 | 40046.7 | +9% || | | | | -| Geomean || | | || | | -8% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - single-threaded, SF 0.01

Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -1%

Configuration Overview - click to expand

```diff +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st_s01.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st_s01.json | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 08:43:01 | 2022-07-24 12:04:22 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 0.009999999776482582 | 0.009999999776482582 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 4.8 | 4.8 | +1%˄ || 208.38 | 206.60 | -1%˄ | 0.0026 | -| TPC-H 02 || 3.6 | 3.8 | +5%˄ || 276.64 | 263.16 | -5%˄ | 0.4236 | -| TPC-H 03 || 0.6 | 0.7 | +11%˄ || 1542.38 | 1386.19 | -10%˄ | 0.0000 | -| TPC-H 04 || 0.4 | 0.5 | +9%˄ || 2320.60 | 2133.89 | -8%˄ | 0.0000 | | TPC-H 05 || 1.0 | 1.0 | +3%˄ || 1000.25 | 971.81 | -3%˄ | 0.0002 | | TPC-H 06 || 0.2 | 0.2 | -1%˄ || 5670.99 | 5714.13 | +1%˄ | 0.0688 | -| TPC-H 07 || 9.9 | 10.9 | +10%˄ || 100.64 | 91.86 | -9%˄ | 0.2114 | | TPC-H 08 || 15.4 | 16.0 | +4%˄ || 64.99 | 62.53 | -4%˄ | 0.0227 | +| TPC-H 09 || 3.7 | 3.1 | -16%˄ || 271.65 | 323.53 | +19%˄ | 0.2420 | | TPC-H 10 || 0.8 | 0.8 | +4%˄ || 1269.20 | 1223.48 | -4%˄ | 0.0000 | | TPC-H 11 || 0.2 | 0.2 | +2%˄ || 4804.31 | 4718.01 | -2%˄ | 0.1601 | | TPC-H 12 || 0.6 | 0.6 | +2%˄ || 1659.81 | 1632.06 | -2%˄ | 0.0000 | +| TPC-H 13 || 2.0 | 1.9 | -5%˄ || 496.80 | 520.62 | +5%˄ | 0.0000 | | TPC-H 14 || 0.3 | 0.3 | -1%˄ || 2912.20 | 2944.67 | +1%˄ | 0.2269 | | TPC-H 15 || 1.1 | 1.1 | -1%˄ || 880.90 | 886.39 | +1%˄ | 0.0140 | | TPC-H 16 || 1.9 | 2.0 | +1%˄ || 513.43 | 510.59 | -1%˄ | 0.0323 | | TPC-H 17 || 0.6 | 0.6 | +2%˄ || 1618.92 | 1592.72 | -2%˄ | 0.8953 | | TPC-H 18 || 1.1 | 1.2 | +1%˄ || 874.49 | 866.72 | -1%˄ | 0.0000 | | TPC-H 19 || 4.8 | 4.9 | +3%˄ || 208.78 | 203.62 | -2%˄ | 0.0000 | | TPC-H 20 || 2.1 | 2.1 | -1%˄ || 476.06 | 482.12 | +1%˄ | 0.5917 | -| TPC-H 21 || 1.5 | 1.5 | +5%˄ || 678.44 | 647.24 | -5%˄ | 0.2686 | | TPC-H 22 || 1.0 | 1.0 | -1%˄ || 993.17 | 1001.69 | +1%˄ | 0.0001 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 57.8 | 59.3 | +3% || | | | | | Geomean || | | || | | -1% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0

Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -4%

Configuration Overview - click to expand

```diff +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt_ordered.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt_ordered.json | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 08:43:30 | 2022-07-24 12:04:50 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | TPC-H 01 || 4757.5 | 4806.9 | +1% || 0.20 | 0.20 | +0% | (run time too short) | | TPC-H 02 || 63.2 | 64.9 | +3% || 14.46 | 14.16 | -2% | 0.0096 | -| TPC-H 03 || 1118.6 | 1212.3 | +8% || 0.88 | 0.82 | -8% | 0.0000 | -| TPC-H 04 || 598.2 | 670.6 | +12% || 1.65 | 1.47 | -11% | 0.0000 | -| TPC-H 05 || 1001.1 | 1073.1 | +7% || 0.98 | 0.92 | -7% | 0.0000 | | TPC-H 06 || 71.8 | 72.2 | +1% || 13.02 | 12.92 | -1% | 0.1608 | -| TPC-H 07 || 462.1 | 483.0 | +5% || 2.13 | 2.05 | -4% | 0.0000 | -| TPC-H 08 || 403.8 | 440.8 | +9% || 2.43 | 2.23 | -8% | 0.0000 | | TPC-H 09 || 2533.0 | 2525.9 | -0% || 0.38 | 0.38 | +0% | (run time too short) | -| TPC-H 10 || 1606.2 | 1683.8 | +5% || 0.62 | 0.58 | -5% | (run time too short) | | TPC-H 11 || 90.3 | 92.2 | +2% || 10.48 | 10.28 | -2% | 0.0005 | -| TPC-H 12 || 549.7 | 594.5 | +8% || 1.80 | 1.67 | -7% | 0.0000 | | TPC-H 13 || 2993.4 | 3081.7 | +3% || 0.32 | 0.32 | +0% | (run time too short) | | TPC-H 14 || 171.4 | 174.4 | +2% || 5.67 | 5.57 | -2% | 0.0001 | | TPC-H 15 || 167.2 | 167.8 | +0% || 5.80 | 5.78 | -0% | 0.4791 | | TPC-H 16 || 774.6 | 772.1 | -0% || 1.27 | 1.28 | +1% | (run time too short) | | TPC-H 17 || 92.8 | 94.9 | +2% || 10.22 | 10.00 | -2% | 0.0000 | | TPC-H 18 || 2597.2 | 2515.9 | -3% || 0.38 | 0.38 | -0% | (run time too short) | | TPC-H 19 || 157.2 | 157.5 | +0% || 6.15 | 6.15 | -0% | 0.7423 | | TPC-H 20 || 253.3 | 252.1 | -0% || 3.87 | 3.88 | +0% | 0.5076 | -| TPC-H 21 || 1189.1 | 1333.5 | +12% || 0.83 | 0.73 | -12% | (run time too short) | -| TPC-H 22 || 154.5 | 174.0 | +13% || 6.25 | 5.57 | -11% | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 21806.1 | 22444.2 | +3% || | | | | | Geomean || | | || | | -4% | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0

Sum of avg. item runtimes: +11% || Geometric mean of throughput changes: -10%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 09:07:16 | 2022-07-24 12:28:39 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ -| TPC-H 01 || 5589.1 | 5692.0 | +2% || 0.47 | 0.42 | -10% | 0.1939 | -| TPC-H 02 || 365.8 | 487.0 | +33% || 0.47 | 0.42 | -10% | 0.1517 | -| TPC-H 03 || 3585.6 | 4063.8 | +13% || 0.47 | 0.42 | -10% | 0.0181 | -| TPC-H 04 || 2381.5 | 2407.5 | +1% || 0.47 | 0.42 | -10% | 0.8814 | -| TPC-H 05 || 4383.4 | 4430.4 | +1% || 0.47 | 0.42 | -10% | 0.8447 | -| TPC-H 06 || 416.5 | 515.3 | +24% || 0.47 | 0.42 | -10% | 0.0581 | -| TPC-H 07 || 2478.5 | 2725.1 | +10% || 0.47 | 0.42 | -10% | 0.2036 | -| TPC-H 08 || 1767.3 | 2278.2 | +29% || 0.47 | 0.42 | -10% | 0.0018 | -| TPC-H 09 || 5812.6 | 6373.1 | +10% || 0.47 | 0.42 | -10% | 0.0106 | -| TPC-H 10 || 4518.8 | 5295.6 | +17% || 0.47 | 0.42 | -10% | 0.0006 | -| TPC-H 11 || 464.8 | 472.0 | +2% || 0.47 | 0.42 | -10% | 0.9208 | -| TPC-H 12 || 2311.2 | 2426.7 | +5% || 0.47 | 0.42 | -10% | 0.5038 | -| TPC-H 13 || 5653.4 | 6212.7 | +10% || 0.47 | 0.42 | -10% | 0.0001 | -| TPC-H 14 || 1041.0 | 1188.3 | +14% || 0.47 | 0.42 | -10% | 0.2182 | -| TPC-H 15 || 479.9 | 461.3 | -4% || 0.47 | 0.42 | -10% | 0.6544 | -| TPC-H 16 || 2204.4 | 2419.4 | +10% || 0.47 | 0.42 | -10% | 0.2153 | -| TPC-H 17 || 514.3 | 705.3 | +37% || 0.47 | 0.42 | -10% | 0.0206 | -| TPC-H 18 || 3411.6 | 3690.7 | +8% || 0.47 | 0.42 | -10% | 0.0114 | -| TPC-H 19 || 728.6 | 932.8 | +28% || 0.47 | 0.42 | -10% | 0.0583 | -| TPC-H 20 || 1278.3 | 1222.7 | -4% || 0.47 | 0.42 | -10% | 0.6280 | -| TPC-H 21 || 5188.8 | 6662.3 | +28% || 0.47 | 0.42 | -10% | 0.0000 | -| TPC-H 22 || 1048.9 | 1008.4 | -4% || 0.47 | 0.42 | -10% | 0.7279 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 55624.3 | 61670.7 | +11% || | | | | -| Geomean || | | || | | -10% | | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - single-threaded

Sum of avg. item runtimes: +4% || Geometric mean of throughput changes: -9%

Configuration Overview - click to expand

```diff +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 09:29:00 | 2022-07-24 12:50:25 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 259.2 | 265.0 | +2%˄ || 3.86 | 3.77 | -2%˄ | 0.0000 | -| 03 || 69.0 | 80.0 | +16%˄ || 14.49 | 12.50 | -14%˄ | 0.0000 | -| 06 || 152.6 | 169.7 | +11%˄ || 6.55 | 5.89 | -10%˄ | 0.0000 | -| 07 || 296.9 | 314.6 | +6%˄ || 3.37 | 3.18 | -6%˄ | 0.0000 | | 09 || 634.0 | 635.6 | +0% || 1.58 | 1.57 | -0% | 0.0052 | -| 10 || 129.2 | 163.3 | +26%˄ || 7.74 | 6.12 | -21%˄ | 0.0000 | -| 13 || 426.0 | 445.1 | +5%˄ || 2.35 | 2.25 | -4%˄ | 0.0000 | -| 15 || 103.4 | 114.9 | +11%˄ || 9.67 | 8.70 | -10%˄ | 0.0000 | -| 16 || 152.3 | 159.4 | +5%˄ || 6.57 | 6.27 | -4%˄ | 0.0000 | -| 17 || 308.4 | 337.8 | +10%˄ || 3.24 | 2.96 | -9%˄ | 0.0000 | -| 19 || 100.0 | 118.5 | +18%˄ || 10.00 | 8.44 | -16%˄ | 0.0000 | -| 25 || 162.8 | 191.2 | +17%˄ || 6.14 | 5.23 | -15%˄ | 0.0000 | | 26 || 134.0 | 136.1 | +2%˄ || 7.46 | 7.35 | -1%˄ | 0.0040 | | 28 || 608.9 | 609.3 | +0% || 1.64 | 1.64 | -0% | 0.6439 | -| 29 || 469.4 | 504.6 | +8%˄ || 2.13 | 1.98 | -7%˄ | 0.0000 | -| 31 || 1268.3 | 1333.1 | +5% || 0.79 | 0.75 | -5% | 0.0000 | -| 32 || 40.5 | 51.0 | +26%˄ || 24.68 | 19.59 | -21%˄ | 0.0000 | -| 34 || 159.0 | 180.8 | +14%˄ || 6.29 | 5.53 | -12%˄ | 0.0000 | -| 35 || 605.1 | 676.2 | +12%˄ || 1.65 | 1.48 | -11%˄ | 0.0000 | -| 37 || 275.9 | 328.3 | +19%˄ || 3.62 | 3.05 | -16%˄ | 0.0000 | | 39a || 1736.9 | 1801.3 | +4% || 0.58 | 0.56 | -4% | 0.0000 | -| 39b || 1728.0 | 1827.3 | +6% || 0.58 | 0.55 | -5% | 0.0000 | | 41 || 288.5 | 286.8 | -1%˄ || 3.47 | 3.49 | +1%˄ | 0.0000 | -| 42 || 80.9 | 99.0 | +22%˄ || 12.36 | 10.10 | -18%˄ | 0.0000 | | 43 || 942.7 | 961.9 | +2% || 1.06 | 1.04 | -2% | 0.0000 | -| 45 || 114.2 | 122.5 | +7%˄ || 8.76 | 8.16 | -7%˄ | 0.0000 | | 48 || 1014.0 | 1047.7 | +3% || 0.99 | 0.95 | -3% | 0.0000 | -| 50 || 109.3 | 129.5 | +19%˄ || 9.15 | 7.72 | -16%˄ | 0.0000 | -| 52 || 78.9 | 97.6 | +24%˄ || 12.67 | 10.24 | -19%˄ | 0.0000 | -| 55 || 75.4 | 94.1 | +25%˄ || 13.26 | 10.63 | -20%˄ | 0.0000 | | 62 || 544.3 | 532.8 | -2%˄ || 1.84 | 1.88 | +2%˄ | 0.0000 | | 65 || 1763.4 | 1777.2 | +1% || 0.57 | 0.56 | -1% | 0.1994 | -| 69 || 119.5 | 151.5 | +27%˄ || 8.37 | 6.60 | -21%˄ | 0.0000 | -| 73 || 76.6 | 95.0 | +24%˄ || 13.05 | 10.53 | -19%˄ | 0.0000 | | 79 || 467.1 | 483.8 | +4%˄ || 2.14 | 2.07 | -3%˄ | 0.0000 | | 81 || 209.4 | 210.0 | +0%˄ || 4.78 | 4.76 | -0%˄ | 0.2427 | -| 82 || 351.1 | 431.9 | +23%˄ || 2.85 | 2.32 | -19%˄ | 0.0000 | -| 83 || 39.8 | 43.9 | +10%˄ || 25.15 | 22.76 | -9%˄ | 0.0000 | | 85 || 134.0 | 133.3 | -1%˄ || 7.46 | 7.50 | +1%˄ | 0.0020 | -| 88 || 617.1 | 820.6 | +33% || 1.62 | 1.22 | -25% | 0.0000 | -| 91 || 15.4 | 18.0 | +17%˄ || 65.08 | 55.67 | -14%˄ | 0.0000 | -| 92 || 34.7 | 40.8 | +17%˄ || 28.79 | 24.52 | -15%˄ | 0.0000 | | 93 || 3878.5 | 3974.1 | +2% || 0.26 | 0.25 | -2% | 0.0011 | -| 94 || 97.9 | 106.2 | +9%˄ || 10.22 | 9.41 | -8%˄ | 0.0000 | | 95 || 9009.6 | 9229.1 | +2% || 0.11 | 0.11 | -2% | ˅ | -| 96 || 57.3 | 76.0 | +33%˄ || 17.44 | 13.15 | -25%˄ | 0.0000 | | 97 || 3242.2 | 3291.9 | +2% || 0.31 | 0.30 | -2% | 0.0044 | | 99 || 1055.5 | 1020.9 | -3% || 0.95 | 0.98 | +3% | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 34237.3 | 35719.1 | +4% || | | | | -| Geomean || | | || | | -9% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | | || ˅ Insufficient number of runs for p-value calculation | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -2%

Configuration Overview - click to expand

```diff +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 09:55:01 | 2022-07-24 13:17:28 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 629.4 | 616.8 | -2% || 0.47 | 0.46 | -2% | 0.8493 | -| 03 || 234.2 | 278.1 | +19% || 0.47 | 0.46 | -2% | 0.3920 | +| 06 || 665.7 | 582.5 | -13% || 0.47 | 0.46 | -2% | 0.3020 | | 07 || 760.8 | 734.6 | -3% || 0.47 | 0.46 | -2% | 0.7257 | | 09 || 1295.6 | 1343.2 | +4% || 0.47 | 0.46 | -2% | 0.5415 | +| 10 || 421.8 | 388.3 | -8% || 0.47 | 0.46 | -1% | 0.6042 | | 13 || 1504.7 | 1453.1 | -3% || 0.47 | 0.46 | -2% | 0.6768 | -| 15 || 380.0 | 504.7 | +33% || 0.47 | 0.46 | -2% | 0.0549 | +| 16 || 824.3 | 778.1 | -6% || 0.47 | 0.46 | -1% | 0.6793 | -| 17 || 1015.2 | 1116.2 | +10% || 0.47 | 0.46 | -2% | 0.3554 | -| 19 || 383.0 | 446.5 | +17% || 0.47 | 0.46 | -1% | 0.2032 | | 25 || 824.4 | 839.7 | +2% || 0.47 | 0.46 | -2% | 0.8918 | +| 26 || 507.5 | 423.4 | -17% || 0.47 | 0.46 | -2% | 0.1430 | +| 28 || 560.7 | 500.2 | -11% || 0.47 | 0.46 | -2% | 0.3560 | -| 29 || 1329.2 | 1432.1 | +8% || 0.47 | 0.46 | -2% | 0.3689 | -| 31 || 1849.9 | 2054.8 | +11% || 0.47 | 0.46 | -2% | 0.1880 | -| 32 || 234.5 | 385.2 | +64% || 0.47 | 0.46 | -2% | 0.0337 | | 34 || 421.0 | 423.6 | +1% || 0.47 | 0.46 | -2% | 0.9584 | +| 35 || 1951.9 | 1727.1 | -12% || 0.47 | 0.46 | -1% | 0.1220 | +| 37 || 565.9 | 516.7 | -9% || 0.47 | 0.46 | -2% | 0.4536 | -| 39a || 2029.8 | 2312.2 | +14% || 0.47 | 0.46 | -2% | 0.0329 | | 39b || 2123.8 | 2075.7 | -2% || 0.47 | 0.46 | -2% | 0.6570 | -| 41 || 2704.4 | 2921.8 | +8% || 0.47 | 0.46 | -2% | 0.1544 | +| 42 || 322.3 | 267.3 | -17% || 0.47 | 0.46 | -1% | 0.3135 | | 43 || 1350.8 | 1344.7 | -0% || 0.47 | 0.46 | -2% | 0.9490 | | 45 || 636.9 | 619.1 | -3% || 0.47 | 0.46 | -1% | 0.8420 | -| 48 || 2534.6 | 2704.2 | +7% || 0.47 | 0.46 | -2% | 0.3100 | -| 50 || 495.5 | 582.3 | +18% || 0.47 | 0.46 | -1% | 0.2477 | +| 52 || 313.5 | 287.7 | -8% || 0.47 | 0.46 | -2% | 0.6104 | | 55 || 286.8 | 282.7 | -1% || 0.47 | 0.46 | -2% | 0.9429 | +| 62 || 838.8 | 789.9 | -6% || 0.47 | 0.46 | -1% | 0.4266 | | 65 || 3623.5 | 3556.1 | -2% || 0.47 | 0.46 | -2% | 0.3915 | -| 69 || 597.2 | 641.7 | +7% || 0.47 | 0.46 | -2% | 0.6243 | -| 73 || 307.7 | 339.9 | +10% || 0.47 | 0.46 | -2% | 0.5927 | +| 79 || 1200.3 | 1137.5 | -5% || 0.47 | 0.46 | -2% | 0.4632 | -| 81 || 728.2 | 804.2 | +10% || 0.47 | 0.46 | -2% | 0.3784 | -| 82 || 720.3 | 881.1 | +22% || 0.47 | 0.46 | -2% | 0.0795 | -| 83 || 309.6 | 336.0 | +9% || 0.47 | 0.46 | -1% | 0.7302 | +| 85 || 744.1 | 659.3 | -11% || 0.47 | 0.46 | -1% | 0.3491 | -| 88 || 655.8 | 787.6 | +20% || 0.47 | 0.46 | -2% | 0.1188 | -| 91 || 194.1 | 211.4 | +9% || 0.47 | 0.46 | -2% | 0.7483 | | 92 || 339.8 | 346.3 | +2% || 0.47 | 0.46 | -2% | 0.9280 | -| 93 || 2302.2 | 2472.3 | +7% || 0.47 | 0.46 | -2% | 0.2183 | | 94 || 631.6 | 605.4 | -4% || 0.47 | 0.46 | -2% | 0.7871 | | 95 || 7935.6 | 7831.1 | -1% || 0.47 | 0.46 | -2% | 0.5190 | +| 96 || 261.5 | 204.8 | -22% || 0.47 | 0.46 | -1% | 0.2457 | | 97 || 4287.2 | 4321.3 | +1% || 0.47 | 0.46 | -2% | 0.8297 | | 99 || 1399.9 | 1366.4 | -2% || 0.47 | 0.46 | -2% | 0.7113 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 56235.4 | 57235.0 | +2% || | | | | | Geomean || | | || | | -2% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - single-threaded

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 10:15:34 | 2022-07-24 13:37:55 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 22.9 | 22.7 | -1% || 4.25 | 4.30 | +1% | 0.1280 | | New-Order || 16.5 | 16.2 | -1% || 47.86 | 48.36 | +1% | 0.0886 | | Order-Status || 1.1 | 1.1 | -1% || 4.26 | 4.30 | +1% | 0.2874 | | Payment || 2.0 | 2.1 | +1% || 45.73 | 46.23 | +1% | 0.1472 | | Stock-Level || 3.5 | 3.6 | +2% || 4.25 | 4.28 | +1% | 0.1108 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 46.0 | 45.7 | -1% || | | | | | Geomean || | | || | | +1% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 10:16:35 | 2022-07-24 13:38:55 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 219.1 | 220.1 | +0% || 4.32 | 4.31 | -0% | 0.5271 | | unsucc.: || 4.3 | 4.3 | -1% || 62.88 | 62.89 | +0% | | | New-Order || 57.3 | 57.4 | +0% || 134.12 | 134.52 | +0% | 0.3170 | | unsucc.: || 4.9 | 4.9 | -1% || 621.96 | 621.40 | -0% | | | Order-Status || 7.6 | 7.6 | -1% || 67.21 | 67.19 | -0% | 0.2578 | | Payment || 9.8 | 9.8 | -0% || 21.57 | 21.42 | -1% | 0.5453 | | unsucc.: || 4.5 | 4.4 | -1% || 700.91 | 700.91 | +0% | | | Stock-Level || 17.3 | 17.5 | +1% || 67.21 | 67.19 | -0% | 0.0017 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 311.2 | 312.5 | +0% || | | | | | Geomean || | | || | | -0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - single-threaded

Sum of avg. item runtimes: +6% || Geometric mean of throughput changes: -12%

Configuration Overview - click to expand

```diff +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 10:36:42 | 2022-07-24 13:59:03 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+----------------------+ -| 10a || 166.8 | 179.3 | +7%˄ || 5.99 | 5.58 | -7%˄ | 0.0000 | -| 10b || 116.4 | 128.7 | +11%˄ || 8.59 | 7.77 | -10%˄ | 0.0000 | | 10c || 322.2 | 317.0 | -2%˄ || 3.10 | 3.15 | +2%˄ | 0.0000 | -| 11a || 15.2 | 20.0 | +31%˄ || 65.67 | 49.96 | -24%˄ | 0.0000 | -| 11b || 14.9 | 19.9 | +33%˄ || 67.10 | 50.35 | -25%˄ | 0.0000 | -| 11c || 20.6 | 25.3 | +23%˄ || 48.52 | 39.54 | -19%˄ | 0.0000 | -| 11d || 23.4 | 28.1 | +20%˄ || 42.65 | 35.63 | -16%˄ | 0.0000 | | 12a || 48.4 | 50.3 | +4%˄ || 20.66 | 19.87 | -4%˄ | 0.0000 | -| 12b || 29.2 | 40.6 | +39%˄ || 34.27 | 24.65 | -28%˄ | 0.0000 | -| 12c || 83.0 | 88.0 | +6%˄ || 12.05 | 11.37 | -6%˄ | 0.0000 | -| 13a || 274.8 | 288.0 | +5%˄ || 3.64 | 3.47 | -5%˄ | 0.0000 | -| 13b || 152.1 | 171.1 | +12%˄ || 6.57 | 5.85 | -11%˄ | 0.0000 | -| 13c || 115.3 | 133.2 | +16%˄ || 8.67 | 7.51 | -13%˄ | 0.0000 | | 13d || 586.7 | 577.5 | -2%˄ || 1.70 | 1.73 | +2%˄ | 0.0000 | | 14a || 415.8 | 431.6 | +4%˄ || 2.41 | 2.32 | -4%˄ | 0.0000 | -| 14b || 232.8 | 254.0 | +9%˄ || 4.29 | 3.94 | -8%˄ | 0.0000 | | 14c || 493.0 | 503.4 | +2%˄ || 2.03 | 1.99 | -2%˄ | 0.0000 | -| 15a || 81.2 | 89.7 | +11%˄ || 12.32 | 11.14 | -10%˄ | 0.0000 | -| 15b || 80.5 | 88.8 | +10%˄ || 12.41 | 11.27 | -9%˄ | 0.0000 | -| 15c || 85.6 | 93.4 | +9%˄ || 11.68 | 10.71 | -8%˄ | 0.0000 | -| 15d || 82.1 | 91.0 | +11%˄ || 12.18 | 10.99 | -10%˄ | 0.0000 | -| 16a || 2226.5 | 2337.0 | +5% || 0.45 | 0.43 | -5% | 0.0000 | | 16b || 3273.4 | 3363.1 | +3% || 0.31 | 0.30 | -3% | 0.0001 | -| 16c || 2328.2 | 2438.2 | +5% || 0.43 | 0.41 | -5% | 0.0000 | -| 16d || 2298.1 | 2401.8 | +5% || 0.44 | 0.42 | -4% | 0.0000 | -| 17a || 580.6 | 623.2 | +7%˄ || 1.72 | 1.60 | -7%˄ | (run time too short) | -| 17b || 457.9 | 501.6 | +10%˄ || 2.18 | 1.99 | -9%˄ | 0.0000 | -| 17c || 436.9 | 480.7 | +10%˄ || 2.29 | 2.08 | -9%˄ | 0.0000 | -| 17d || 516.8 | 562.7 | +9%˄ || 1.93 | 1.78 | -8%˄ | 0.0000 | | 17e || 1582.0 | 1628.4 | +3% || 0.63 | 0.61 | -3% | 0.0000 | -| 17f || 938.5 | 991.8 | +6% || 1.07 | 1.01 | -5% | 0.0000 | -| 18a || 598.0 | 646.0 | +8%˄ || 1.67 | 1.55 | -7%˄ | 0.0000 | -| 18b || 112.3 | 124.7 | +11%˄ || 8.91 | 8.02 | -10%˄ | 0.0000 | -| 18c || 557.9 | 589.1 | +6%˄ || 1.79 | 1.70 | -5%˄ | 0.0000 | -| 19a || 180.7 | 194.7 | +8%˄ || 5.53 | 5.14 | -7%˄ | 0.0000 | -| 19b || 124.3 | 137.4 | +11%˄ || 8.04 | 7.28 | -10%˄ | 0.0000 | -| 19c || 192.6 | 206.7 | +7%˄ || 5.19 | 4.84 | -7%˄ | 0.0000 | | 19d || 717.3 | 738.1 | +3% || 1.39 | 1.35 | -3% | 0.0000 | -| 1a || 9.0 | 11.9 | +32%˄ || 110.59 | 83.79 | -24%˄ | 0.0000 | -| 1b || 8.5 | 11.4 | +34%˄ || 118.10 | 88.02 | -25%˄ | 0.0000 | -| 1c || 9.5 | 12.5 | +31%˄ || 104.99 | 80.29 | -24%˄ | 0.0000 | -| 1d || 8.5 | 11.4 | +34%˄ || 118.24 | 87.94 | -26%˄ | 0.0000 | | 20a || 390.6 | 407.0 | +4%˄ || 2.56 | 2.46 | -4%˄ | 0.0000 | -| 20b || 228.2 | 245.8 | +8%˄ || 4.38 | 4.07 | -7%˄ | 0.0000 | -| 20c || 219.8 | 237.5 | +8%˄ || 4.55 | 4.21 | -7%˄ | 0.0000 | -| 21a || 35.1 | 48.6 | +38%˄ || 28.47 | 20.59 | -28%˄ | 0.0000 | -| 21b || 19.1 | 24.2 | +27%˄ || 52.48 | 41.33 | -21%˄ | 0.0000 | -| 21c || 34.7 | 48.1 | +39%˄ || 28.86 | 20.81 | -28%˄ | 0.0000 | -| 22a || 195.6 | 216.8 | +11%˄ || 5.11 | 4.61 | -10%˄ | 0.0000 | -| 22b || 142.5 | 156.8 | +10%˄ || 7.02 | 6.38 | -9%˄ | 0.0000 | -| 22c || 493.0 | 517.9 | +5%˄ || 2.03 | 1.93 | -5%˄ | 0.0000 | -| 22d || 895.6 | 947.5 | +6% || 1.12 | 1.06 | -5% | 0.0000 | -| 23a || 42.8 | 54.9 | +28%˄ || 23.39 | 18.21 | -22%˄ | 0.0000 | -| 23b || 43.9 | 55.5 | +26%˄ || 22.76 | 18.01 | -21%˄ | 0.0000 | -| 23c || 85.5 | 94.0 | +10%˄ || 11.69 | 10.63 | -9%˄ | 0.0000 | -| 24a || 144.1 | 169.6 | +18%˄ || 6.94 | 5.90 | -15%˄ | 0.0000 | -| 24b || 122.8 | 148.4 | +21%˄ || 8.14 | 6.74 | -17%˄ | 0.0000 | -| 25a || 276.5 | 292.9 | +6%˄ || 3.62 | 3.41 | -6%˄ | 0.0000 | -| 25b || 113.3 | 136.9 | +21%˄ || 8.82 | 7.30 | -17%˄ | 0.0000 | | 25c || 908.2 | 945.2 | +4% || 1.10 | 1.06 | -4% | 0.0000 | -| 26a || 163.5 | 180.8 | +11%˄ || 6.12 | 5.53 | -10%˄ | 0.0000 | -| 26b || 131.0 | 148.4 | +13%˄ || 7.63 | 6.74 | -12%˄ | 0.0000 | -| 26c || 232.5 | 249.7 | +7%˄ || 4.30 | 4.01 | -7%˄ | 0.0000 | -| 27a || 22.6 | 27.9 | +24%˄ || 44.28 | 35.80 | -19%˄ | 0.0000 | -| 27b || 22.2 | 27.4 | +23%˄ || 44.98 | 36.55 | -19%˄ | 0.0000 | -| 27c || 34.7 | 48.2 | +39%˄ || 28.81 | 20.73 | -28%˄ | 0.0000 | -| 28a || 290.0 | 306.5 | +6%˄ || 3.45 | 3.26 | -5%˄ | 0.0000 | -| 28b || 37.2 | 55.4 | +49%˄ || 26.90 | 18.05 | -33%˄ | 0.0000 | -| 28c || 328.1 | 347.2 | +6%˄ || 3.05 | 2.88 | -5%˄ | 0.0000 | -| 29a || 101.8 | 133.3 | +31%˄ || 9.82 | 7.50 | -24%˄ | 0.0000 | -| 29b || 76.2 | 106.6 | +40%˄ || 13.12 | 9.38 | -29%˄ | 0.0000 | -| 29c || 148.5 | 176.2 | +19%˄ || 6.73 | 5.68 | -16%˄ | 0.0000 | -| 2a || 46.1 | 51.2 | +11%˄ || 21.67 | 19.53 | -10%˄ | 0.0000 | -| 2b || 37.4 | 42.5 | +14%˄ || 26.70 | 23.51 | -12%˄ | 0.0000 | -| 2c || 21.8 | 27.0 | +24%˄ || 45.93 | 37.02 | -19%˄ | 0.0000 | -| 2d || 64.6 | 70.2 | +9%˄ || 15.47 | 14.23 | -8%˄ | 0.0000 | -| 30a || 148.6 | 166.2 | +12%˄ || 6.73 | 6.02 | -11%˄ | 0.0000 | -| 30b || 127.5 | 144.4 | +13%˄ || 7.84 | 6.92 | -12%˄ | 0.0000 | -| 30c || 377.1 | 404.3 | +7%˄ || 2.65 | 2.47 | -7%˄ | 0.0000 | -| 31a || 146.0 | 172.4 | +18%˄ || 6.85 | 5.80 | -15%˄ | 0.0000 | -| 31b || 124.6 | 150.9 | +21%˄ || 8.03 | 6.63 | -17%˄ | 0.0000 | -| 31c || 170.4 | 197.0 | +16%˄ || 5.87 | 5.08 | -13%˄ | 0.0000 | -| 32a || 12.9 | 17.1 | +32%˄ || 77.21 | 58.64 | -24%˄ | 0.0000 | -| 32b || 30.9 | 36.9 | +19%˄ || 32.33 | 27.10 | -16%˄ | 0.0000 | -| 33a || 19.5 | 26.0 | +34%˄ || 51.34 | 38.43 | -25%˄ | 0.0000 | -| 33b || 18.9 | 25.5 | +35%˄ || 52.94 | 39.24 | -26%˄ | 0.0000 | -| 33c || 22.8 | 29.3 | +29%˄ || 43.94 | 34.14 | -22%˄ | 0.0000 | | 3a || 122.7 | 127.5 | +4%˄ || 8.15 | 7.84 | -4%˄ | 0.0000 | -| 3b || 12.7 | 16.3 | +29%˄ || 78.91 | 61.37 | -22%˄ | 0.0000 | -| 3c || 436.1 | 458.5 | +5%˄ || 2.29 | 2.18 | -5%˄ | 0.0000 | +| 4a || 112.5 | 107.0 | -5%˄ || 8.89 | 9.35 | +5%˄ | 0.0000 | -| 4b || 11.5 | 15.3 | +32%˄ || 86.71 | 65.55 | -24%˄ | 0.0000 | | 4c || 124.0 | 123.2 | -1%˄ || 8.07 | 8.11 | +1%˄ | 0.0015 | | 5a || 52.5 | 54.2 | +3%˄ || 19.04 | 18.43 | -3%˄ | 0.0000 | +| 5b || 134.8 | 125.4 | -7%˄ || 7.42 | 7.97 | +7%˄ | 0.0000 | -| 5c || 185.1 | 194.3 | +5%˄ || 5.40 | 5.15 | -5%˄ | 0.0000 | -| 6a || 103.5 | 119.3 | +15%˄ || 9.66 | 8.38 | -13%˄ | 0.0000 | -| 6b || 115.5 | 130.6 | +13%˄ || 8.66 | 7.65 | -12%˄ | 0.0000 | -| 6c || 99.0 | 114.1 | +15%˄ || 10.10 | 8.77 | -13%˄ | 0.0000 | -| 6d || 527.4 | 558.0 | +6%˄ || 1.90 | 1.79 | -5%˄ | 0.0000 | -| 6e || 103.5 | 119.1 | +15%˄ || 9.66 | 8.40 | -13%˄ | 0.0000 | | 6f || 696.7 | 720.6 | +3% || 1.44 | 1.39 | -3% | 0.0000 | -| 7a || 53.3 | 78.2 | +47%˄ || 18.77 | 12.79 | -32%˄ | 0.0000 | -| 7b || 49.6 | 74.8 | +51%˄ || 20.17 | 13.38 | -34%˄ | 0.0000 | | 7c || 582.6 | 601.2 | +3%˄ || 1.72 | 1.66 | -3%˄ | 0.0000 | -| 8a || 38.0 | 43.2 | +14%˄ || 26.30 | 23.14 | -12%˄ | 0.0000 | -| 8b || 35.8 | 40.9 | +14%˄ || 27.95 | 24.44 | -13%˄ | 0.0000 | -| 8c || 1858.3 | 1955.7 | +5% || 0.54 | 0.51 | -5% | 0.0000 | -| 8d || 303.8 | 347.6 | +14%˄ || 3.29 | 2.88 | -13%˄ | 0.0000 | | 9a || 187.1 | 193.2 | +3%˄ || 5.35 | 5.18 | -3%˄ | 0.0000 | | 9b || 104.5 | 109.0 | +4%˄ || 9.57 | 9.17 | -4%˄ | 0.0000 | | 9c || 269.1 | 274.1 | +2%˄ || 3.72 | 3.65 | -2%˄ | 0.0000 | | 9d || 471.9 | 479.8 | +2%˄ || 2.12 | 2.08 | -2%˄ | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+----------------------+ -| Sum || 34738.0 | 36928.1 | +6% || | | | | -| Geomean || | | || | | -12% | | +---------++----------+---------+--------++----------+----------+--------+----------------------+ | Notes || ˄ Execution stopped due to max runs reached | +---------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +7% || Geometric mean of throughput changes: -7%

Configuration Overview - click to expand

```diff +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 11:18:48 | 2022-07-24 14:43:25 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ -| 10a || 190.7 | 231.1 | +21% || 0.51 | 0.47 | -7% | 0.2552 | -| 10b || 128.6 | 137.5 | +7% || 0.51 | 0.47 | -7% | 0.5953 | -| 10c || 439.8 | 456.9 | +4% || 0.51 | 0.47 | -7% | 0.6675 | -| 11a || 58.6 | 60.4 | +3% || 0.51 | 0.47 | -7% | 0.9166 | -| 11b || 47.7 | 49.8 | +4% || 0.51 | 0.47 | -7% | 0.8512 | -| 11c || 73.5 | 72.9 | -1% || 0.51 | 0.47 | -7% | 0.9686 | -| 11d || 64.6 | 102.6 | +59% || 0.51 | 0.47 | -7% | 0.0218 | -| 12a || 190.1 | 192.4 | +1% || 0.51 | 0.47 | -7% | 0.9270 | -| 12b || 60.1 | 87.3 | +45% || 0.51 | 0.47 | -7% | 0.1445 | -| 12c || 282.4 | 336.5 | +19% || 0.51 | 0.47 | -7% | 0.0492 | +| 13a || 636.9 | 588.7 | -8% || 0.51 | 0.47 | -7% | 0.2447 | -| 13b || 231.4 | 301.4 | +30% || 0.51 | 0.47 | -7% | 0.0227 | -| 13c || 273.0 | 262.3 | -4% || 0.51 | 0.47 | -7% | 0.7584 | +| 13d || 1045.9 | 993.4 | -5% || 0.51 | 0.47 | -7% | 0.3326 | +| 14a || 802.0 | 762.3 | -5% || 0.51 | 0.47 | -7% | 0.4453 | -| 14b || 442.7 | 516.9 | +17% || 0.51 | 0.47 | -7% | 0.0458 | -| 14c || 901.2 | 990.2 | +10% || 0.51 | 0.47 | -7% | 0.0873 | -| 15a || 139.7 | 164.6 | +18% || 0.51 | 0.47 | -7% | 0.1838 | -| 15b || 117.7 | 135.7 | +15% || 0.51 | 0.47 | -7% | 0.2115 | -| 15c || 151.5 | 155.2 | +2% || 0.51 | 0.47 | -7% | 0.8600 | +| 15d || 230.5 | 195.3 | -15% || 0.51 | 0.47 | -7% | 0.3155 | -| 16a || 2499.7 | 2706.4 | +8% || 0.51 | 0.47 | -7% | 0.0009 | -| 16b || 3777.2 | 3923.8 | +4% || 0.51 | 0.47 | -7% | 0.0158 | -| 16c || 2579.8 | 2806.2 | +9% || 0.51 | 0.47 | -7% | 0.0001 | -| 16d || 2608.6 | 2800.2 | +7% || 0.51 | 0.47 | -7% | 0.0010 | -| 17a || 854.0 | 824.6 | -3% || 0.51 | 0.47 | -7% | 0.6178 | -| 17b || 564.4 | 640.1 | +13% || 0.51 | 0.47 | -7% | 0.0558 | -| 17c || 506.1 | 571.3 | +13% || 0.51 | 0.47 | -7% | 0.1041 | -| 17d || 570.8 | 582.8 | +2% || 0.51 | 0.47 | -7% | 0.7635 | -| 17e || 1488.6 | 1554.2 | +4% || 0.51 | 0.47 | -7% | 0.1529 | -| 17f || 1009.2 | 1052.1 | +4% || 0.51 | 0.47 | -7% | 0.3224 | -| 18a || 729.9 | 777.0 | +6% || 0.51 | 0.47 | -7% | 0.2976 | -| 18b || 139.6 | 160.2 | +15% || 0.51 | 0.47 | -7% | 0.2965 | +| 18c || 701.2 | 637.7 | -9% || 0.51 | 0.47 | -7% | 0.1395 | +| 19a || 498.1 | 459.6 | -8% || 0.51 | 0.47 | -7% | 0.4202 | -| 19b || 351.6 | 340.6 | -3% || 0.51 | 0.47 | -7% | 0.7682 | -| 19c || 480.6 | 520.0 | +8% || 0.51 | 0.47 | -7% | 0.3571 | -| 19d || 1201.1 | 1254.7 | +4% || 0.51 | 0.47 | -7% | 0.3100 | -| 1a || 31.2 | 37.4 | +20% || 0.51 | 0.47 | -7% | 0.5009 | -| 1b || 23.3 | 42.1 | +81% || 0.51 | 0.47 | -7% | 0.0550 | -| 1c || 36.7 | 41.5 | +13% || 0.51 | 0.47 | -7% | 0.6104 | +| 1d || 40.2 | 33.2 | -17% || 0.51 | 0.47 | -7% | 0.5945 | +| 20a || 631.2 | 599.8 | -5% || 0.51 | 0.47 | -7% | 0.5225 | -| 20b || 387.0 | 422.8 | +9% || 0.51 | 0.47 | -7% | 0.2961 | +| 20c || 435.8 | 397.2 | -9% || 0.51 | 0.47 | -7% | 0.3060 | -| 21a || 89.7 | 98.7 | +10% || 0.51 | 0.47 | -7% | 0.6725 | -| 21b || 61.2 | 64.1 | +5% || 0.51 | 0.47 | -7% | 0.8076 | -| 21c || 55.2 | 67.6 | +22% || 0.51 | 0.47 | -7% | 0.1369 | -| 22a || 523.3 | 616.5 | +18% || 0.51 | 0.47 | -7% | 0.0620 | -| 22b || 318.9 | 371.2 | +16% || 0.51 | 0.47 | -7% | 0.1513 | -| 22c || 805.6 | 930.4 | +15% || 0.51 | 0.47 | -7% | 0.0190 | -| 22d || 1191.3 | 1321.2 | +11% || 0.51 | 0.47 | -7% | 0.0208 | -| 23a || 152.4 | 181.6 | +19% || 0.51 | 0.47 | -7% | 0.2363 | +| 23b || 126.0 | 114.2 | -9% || 0.51 | 0.47 | -7% | 0.5072 | +| 23c || 196.3 | 163.1 | -17% || 0.51 | 0.47 | -7% | 0.1907 | -| 24a || 236.6 | 263.8 | +12% || 0.51 | 0.47 | -7% | 0.4098 | -| 24b || 151.0 | 182.7 | +21% || 0.51 | 0.47 | -7% | 0.2111 | -| 25a || 535.9 | 537.7 | +0% || 0.51 | 0.47 | -7% | 0.9692 | -| 25b || 128.2 | 139.1 | +8% || 0.51 | 0.47 | -7% | 0.5601 | -| 25c || 946.6 | 1077.5 | +14% || 0.51 | 0.47 | -7% | 0.0201 | -| 26a || 319.4 | 305.6 | -4% || 0.51 | 0.47 | -7% | 0.6483 | -| 26b || 199.1 | 200.2 | +1% || 0.51 | 0.47 | -7% | 0.9618 | +| 26c || 503.6 | 469.3 | -7% || 0.51 | 0.47 | -7% | 0.4538 | +| 27a || 95.9 | 83.7 | -13% || 0.51 | 0.47 | -7% | 0.3425 | +| 27b || 79.4 | 71.4 | -10% || 0.51 | 0.47 | -7% | 0.5022 | +| 27c || 91.0 | 75.3 | -17% || 0.51 | 0.47 | -7% | 0.3580 | -| 28a || 692.8 | 789.7 | +14% || 0.51 | 0.47 | -7% | 0.0807 | -| 28b || 169.6 | 173.0 | +2% || 0.51 | 0.47 | -7% | 0.8698 | -| 28c || 690.2 | 803.5 | +16% || 0.51 | 0.47 | -7% | 0.0237 | -| 29a || 242.9 | 250.5 | +3% || 0.51 | 0.47 | -7% | 0.8390 | -| 29b || 130.7 | 157.3 | +20% || 0.51 | 0.47 | -7% | 0.1271 | -| 29c || 224.4 | 237.1 | +6% || 0.51 | 0.47 | -7% | 0.7241 | +| 2a || 198.7 | 181.5 | -9% || 0.51 | 0.47 | -7% | 0.5758 | -| 2b || 123.4 | 139.7 | +13% || 0.51 | 0.47 | -7% | 0.3712 | +| 2c || 80.5 | 69.7 | -13% || 0.51 | 0.47 | -7% | 0.4251 | -| 2d || 196.4 | 194.4 | -1% || 0.51 | 0.47 | -7% | 0.9406 | -| 30a || 265.3 | 339.7 | +28% || 0.51 | 0.47 | -7% | 0.0310 | -| 30b || 197.8 | 261.2 | +32% || 0.51 | 0.47 | -7% | 0.0414 | -| 30c || 709.7 | 794.5 | +12% || 0.51 | 0.47 | -7% | 0.1208 | -| 31a || 308.6 | 328.7 | +7% || 0.51 | 0.47 | -7% | 0.6117 | +| 31b || 171.9 | 161.6 | -6% || 0.51 | 0.47 | -7% | 0.6086 | -| 31c || 310.5 | 349.1 | +12% || 0.51 | 0.47 | -7% | 0.2257 | -| 32a || 48.1 | 46.7 | -3% || 0.51 | 0.47 | -7% | 0.9163 | -| 32b || 140.4 | 147.9 | +5% || 0.51 | 0.47 | -7% | 0.7810 | -| 33a || 58.5 | 94.8 | +62% || 0.51 | 0.47 | -7% | 0.0726 | -| 33b || 71.3 | 69.4 | -3% || 0.51 | 0.47 | -7% | 0.9076 | +| 33c || 102.3 | 86.5 | -15% || 0.51 | 0.47 | -7% | 0.4034 | -| 3a || 349.9 | 344.9 | -1% || 0.51 | 0.47 | -7% | 0.8871 | -| 3b || 52.9 | 62.5 | +18% || 0.51 | 0.47 | -7% | 0.4881 | -| 3c || 530.8 | 551.8 | +4% || 0.51 | 0.47 | -7% | 0.5676 | -| 4a || 210.0 | 241.3 | +15% || 0.51 | 0.47 | -7% | 0.3288 | -| 4b || 51.9 | 49.8 | -4% || 0.51 | 0.47 | -7% | 0.9003 | -| 4c || 205.0 | 223.3 | +9% || 0.51 | 0.47 | -7% | 0.4856 | +| 5a || 180.1 | 168.5 | -6% || 0.51 | 0.47 | -7% | 0.6853 | -| 5b || 182.8 | 192.6 | +5% || 0.51 | 0.47 | -7% | 0.6177 | -| 5c || 212.3 | 232.2 | +9% || 0.51 | 0.47 | -7% | 0.4545 | -| 6a || 88.0 | 122.7 | +39% || 0.51 | 0.47 | -7% | 0.0435 | +| 6b || 184.0 | 151.4 | -18% || 0.51 | 0.47 | -7% | 0.1690 | -| 6c || 102.1 | 122.6 | +20% || 0.51 | 0.47 | -7% | 0.3546 | -| 6d || 507.3 | 596.6 | +18% || 0.51 | 0.47 | -7% | 0.0228 | -| 6e || 89.3 | 122.1 | +37% || 0.51 | 0.47 | -7% | 0.0439 | -| 6f || 969.2 | 989.2 | +2% || 0.51 | 0.47 | -7% | 0.6224 | -| 7a || 103.7 | 140.5 | +36% || 0.51 | 0.47 | -7% | 0.0472 | -| 7b || 95.8 | 100.1 | +4% || 0.51 | 0.47 | -7% | 0.7665 | -| 7c || 798.9 | 930.7 | +16% || 0.51 | 0.47 | -7% | 0.0051 | -| 8a || 127.2 | 157.8 | +24% || 0.51 | 0.47 | -7% | 0.1900 | -| 8b || 122.0 | 149.4 | +22% || 0.51 | 0.47 | -7% | 0.2366 | -| 8c || 2107.0 | 2364.4 | +12% || 0.51 | 0.47 | -7% | 0.0000 | -| 8d || 556.9 | 668.3 | +20% || 0.51 | 0.47 | -7% | 0.0049 | -| 9a || 418.6 | 500.4 | +20% || 0.51 | 0.47 | -7% | 0.0335 | -| 9b || 295.8 | 378.0 | +28% || 0.51 | 0.47 | -7% | 0.0208 | -| 9c || 573.3 | 656.1 | +14% || 0.51 | 0.47 | -7% | 0.0381 | -| 9d || 893.7 | 960.7 | +7% || 0.51 | 0.47 | -7% | 0.1950 | +---------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 50603.1 | 54167.9 | +7% || | | | | -| Geomean || | | || | | -7% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

[mweisgut]

[2] Master vs. Memory Tracking Disabled (Iteration 1)

System Command:

../scripts/benchmark_all.sh 1c24c5f49719e589324037c5037866a7388a034c 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8

nemea - click to expand

| property | value | | -- | -- | | Hostname | nemea | | CPU | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | Memory | 1.4TB | | numactl | nodebind: 1 | | numactl | membind: 1 |

Commit Info and Build Time	commit	date	message	build time
1c24c5f49	21.07.2022 21:52	Ensure correct results of EXTRACT expressions (#2482)	real 332.61 user 2798.84 sys 79.02
5acb2f2a0	22.07.2022 10:35	disabled tracking	real 330.22 user 2783.27 sys 78.37

hyriseBenchmarkTPCH - single-threaded, SF 10.0

Sum of avg. item runtimes: +6% || Geometric mean of throughput changes: -5%

Configuration Overview - click to expand

```diff +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 08:23:46 | 2022-07-23 11:09:20 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ +| TPC-H 01 || 5827.3 | 5139.3 | -12% || 0.17 | 0.19 | +13% | 0.0000 | | TPC-H 02 || 42.7 | 43.3 | +1%˄ || 23.39 | 23.11 | -1%˄ | 0.1567 | -| TPC-H 03 || 2208.7 | 2630.0 | +19% || 0.45 | 0.38 | -16% | 0.0000 | -| TPC-H 04 || 1399.1 | 1512.0 | +8% || 0.71 | 0.66 | -7% | 0.0000 | -| TPC-H 05 || 3022.5 | 3318.7 | +10% || 0.33 | 0.30 | -9% | 0.0000 | | TPC-H 06 || 156.2 | 150.4 | -4%˄ || 6.40 | 6.65 | +4%˄ | 0.0000 | -| TPC-H 07 || 919.6 | 970.9 | +6% || 1.09 | 1.03 | -5% | 0.0000 | -| TPC-H 08 || 701.0 | 780.1 | +11% || 1.43 | 1.28 | -10% | 0.0000 | | TPC-H 09 || 5172.5 | 5375.4 | +4% || 0.19 | 0.19 | -4% | 0.0000 | -| TPC-H 10 || 2876.4 | 3105.2 | +8% || 0.35 | 0.32 | -7% | 0.0000 | | TPC-H 11 || 72.1 | 73.0 | +1%˄ || 13.88 | 13.71 | -1%˄ | 0.0982 | -| TPC-H 12 || 971.4 | 1058.7 | +9% || 1.03 | 0.94 | -8% | 0.0000 | -| TPC-H 13 || 4635.8 | 5376.6 | +16% || 0.22 | 0.19 | -14% | 0.0000 | | TPC-H 14 || 416.4 | 433.2 | +4%˄ || 2.40 | 2.31 | -4%˄ | 0.0000 | | TPC-H 15 || 188.5 | 189.5 | +1%˄ || 5.31 | 5.28 | -1%˄ | 0.0000 | | TPC-H 16 || 628.2 | 631.4 | +1% || 1.59 | 1.58 | -0% | 0.0013 | | TPC-H 17 || 213.7 | 216.8 | +1%˄ || 4.68 | 4.61 | -1%˄ | 0.0000 | | TPC-H 18 || 1517.2 | 1493.8 | -2% || 0.66 | 0.67 | +2% | 0.0038 | | TPC-H 19 || 266.0 | 269.8 | +1%˄ || 3.76 | 3.71 | -1%˄ | 0.0000 | | TPC-H 20 || 397.3 | 402.1 | +1%˄ || 2.52 | 2.49 | -1%˄ | 0.0001 | -| TPC-H 21 || 4693.7 | 5312.0 | +13% || 0.21 | 0.19 | -12% | 0.0000 | -| TPC-H 22 || 422.4 | 473.3 | +12%˄ || 2.37 | 2.11 | -11%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 36748.7 | 38955.4 | +6% || | | | | -| Geomean || | | || | | -5% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - single-threaded, SF 0.01

Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -2%

Configuration Overview - click to expand

```diff +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st_s01.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st_s01.json | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 08:43:01 | 2022-07-23 11:30:30 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 0.009999999776482582 | 0.009999999776482582 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 4.8 | 4.9 | +1%˄ || 208.38 | 205.96 | -1%˄ | 0.0011 | -| TPC-H 02 || 3.6 | 3.8 | +5%˄ || 276.64 | 263.65 | -5%˄ | 0.4395 | | TPC-H 03 || 0.6 | 0.7 | +4%˄ || 1542.38 | 1482.76 | -4%˄ | 0.0011 | | TPC-H 04 || 0.4 | 0.4 | +2%˄ || 2320.60 | 2271.11 | -2%˄ | 0.0000 | | TPC-H 05 || 1.0 | 1.0 | +1%˄ || 1000.25 | 994.44 | -1%˄ | 0.4483 | | TPC-H 06 || 0.2 | 0.2 | +1%˄ || 5670.99 | 5635.68 | -1%˄ | 0.1385 | -| TPC-H 07 || 9.9 | 10.8 | +8%˄ || 100.64 | 92.87 | -8%˄ | 0.2727 | | TPC-H 08 || 15.4 | 15.7 | +2%˄ || 64.99 | 63.58 | -2%˄ | 0.2625 | -| TPC-H 09 || 3.7 | 3.9 | +5%˄ || 271.65 | 257.94 | -5%˄ | 0.7277 | | TPC-H 10 || 0.8 | 0.8 | +0%˄ || 1269.20 | 1266.15 | -0%˄ | 0.5747 | | TPC-H 11 || 0.2 | 0.2 | +2%˄ || 4804.31 | 4709.33 | -2%˄ | 0.0990 | | TPC-H 12 || 0.6 | 0.6 | +2%˄ || 1659.81 | 1628.97 | -2%˄ | 0.0000 | | TPC-H 13 || 2.0 | 2.1 | +2%˄ || 496.80 | 485.24 | -2%˄ | 0.0000 | | TPC-H 14 || 0.3 | 0.3 | +2%˄ || 2912.20 | 2862.08 | -2%˄ | 0.0724 | | TPC-H 15 || 1.1 | 1.1 | +0%˄ || 880.90 | 877.85 | -0%˄ | 0.2279 | | TPC-H 16 || 1.9 | 1.9 | +0%˄ || 513.43 | 512.95 | -0%˄ | 0.7168 | -| TPC-H 17 || 0.6 | 0.7 | +7%˄ || 1618.92 | 1509.60 | -7%˄ | 0.5676 | | TPC-H 18 || 1.1 | 1.1 | -0%˄ || 874.49 | 877.74 | +0%˄ | 0.0032 | | TPC-H 19 || 4.8 | 4.8 | -0%˄ || 208.78 | 208.90 | +0%˄ | 0.5153 | | TPC-H 20 || 2.1 | 2.1 | +1%˄ || 476.06 | 473.33 | -1%˄ | 0.7245 | +| TPC-H 21 || 1.5 | 1.4 | -4%˄ || 678.44 | 710.39 | +5%˄ | 0.3035 | | TPC-H 22 || 1.0 | 1.0 | -2%˄ || 993.17 | 1009.69 | +2%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 57.8 | 59.5 | +3% || | | | | | Geomean || | | || | | -2% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0

Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -3%

Configuration Overview - click to expand

```diff +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt_ordered.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt_ordered.json | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 08:43:30 | 2022-07-23 11:31:01 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | TPC-H 01 || 4757.5 | 4788.3 | +1% || 0.20 | 0.20 | +0% | (run time too short) | | TPC-H 02 || 63.2 | 63.2 | +0% || 14.46 | 14.50 | +0% | 0.9595 | -| TPC-H 03 || 1118.6 | 1194.3 | +7% || 0.88 | 0.82 | -8% | (run time too short) | -| TPC-H 04 || 598.2 | 655.8 | +10% || 1.65 | 1.50 | -9% | 0.0000 | -| TPC-H 05 || 1001.1 | 1071.1 | +7% || 0.98 | 0.92 | -7% | (run time too short) | | TPC-H 06 || 71.8 | 71.8 | +0% || 13.02 | 13.02 | +0% | 0.8758 | | TPC-H 07 || 462.1 | 474.7 | +3% || 2.13 | 2.08 | -2% | 0.0000 | -| TPC-H 08 || 403.8 | 431.3 | +7% || 2.43 | 2.28 | -6% | 0.0000 | | TPC-H 09 || 2533.0 | 2508.9 | -1% || 0.38 | 0.38 | +0% | (run time too short) | -| TPC-H 10 || 1606.2 | 1680.6 | +5% || 0.62 | 0.58 | -5% | (run time too short) | | TPC-H 11 || 90.3 | 90.8 | +1% || 10.48 | 10.45 | -0% | 0.3575 | -| TPC-H 12 || 549.7 | 585.0 | +6% || 1.80 | 1.68 | -6% | 0.0000 | | TPC-H 13 || 2993.4 | 3065.1 | +2% || 0.32 | 0.32 | +0% | (run time too short) | | TPC-H 14 || 171.4 | 173.4 | +1% || 5.67 | 5.60 | -1% | 0.0099 | | TPC-H 15 || 167.2 | 167.7 | +0% || 5.80 | 5.78 | -0% | 0.5674 | | TPC-H 16 || 774.6 | 785.6 | +1% || 1.27 | 1.25 | -1% | (run time too short) | | TPC-H 17 || 92.8 | 95.0 | +2% || 10.22 | 10.00 | -2% | 0.0000 | | TPC-H 18 || 2597.2 | 2585.1 | -0% || 0.38 | 0.38 | +0% | 0.8013 | | TPC-H 19 || 157.2 | 156.1 | -1% || 6.15 | 6.20 | +1% | 0.2215 | | TPC-H 20 || 253.3 | 250.4 | -1% || 3.87 | 3.92 | +1% | 0.0798 | -| TPC-H 21 || 1189.1 | 1327.0 | +12% || 0.83 | 0.75 | -10% | 0.0000 | -| TPC-H 22 || 154.5 | 173.7 | +12% || 6.25 | 5.58 | -11% | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 21806.1 | 22394.8 | +3% || | | | | | Geomean || | | || | | -3% | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0

Sum of avg. item runtimes: +9% || Geometric mean of throughput changes: -9%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 09:07:16 | 2022-07-23 11:54:49 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ -| TPC-H 01 || 5589.1 | 5725.6 | +2% || 0.47 | 0.42 | -9% | 0.1230 | -| TPC-H 02 || 365.8 | 369.3 | +1% || 0.47 | 0.43 | -9% | 0.9632 | -| TPC-H 03 || 3585.6 | 3607.1 | +1% || 0.47 | 0.43 | -9% | 0.9068 | -| TPC-H 04 || 2381.5 | 2657.5 | +12% || 0.47 | 0.43 | -9% | 0.1445 | -| TPC-H 05 || 4383.4 | 4489.5 | +2% || 0.47 | 0.43 | -9% | 0.6508 | -| TPC-H 06 || 416.5 | 492.4 | +18% || 0.47 | 0.43 | -9% | 0.1204 | -| TPC-H 07 || 2478.5 | 2744.6 | +11% || 0.47 | 0.43 | -9% | 0.1530 | -| TPC-H 08 || 1767.3 | 2347.3 | +33% || 0.47 | 0.42 | -9% | 0.0003 | -| TPC-H 09 || 5812.6 | 6650.6 | +14% || 0.47 | 0.42 | -9% | 0.0005 | -| TPC-H 10 || 4518.8 | 5275.2 | +17% || 0.47 | 0.43 | -9% | 0.0007 | -| TPC-H 11 || 464.8 | 522.0 | +12% || 0.47 | 0.43 | -9% | 0.4784 | -| TPC-H 12 || 2311.2 | 2434.1 | +5% || 0.47 | 0.43 | -9% | 0.4881 | -| TPC-H 13 || 5653.4 | 6221.5 | +10% || 0.47 | 0.42 | -9% | 0.0003 | -| TPC-H 14 || 1041.0 | 1048.7 | +1% || 0.47 | 0.43 | -9% | 0.9429 | -| TPC-H 15 || 479.9 | 462.4 | -4% || 0.47 | 0.43 | -9% | 0.7148 | -| TPC-H 16 || 2204.4 | 2397.6 | +9% || 0.47 | 0.42 | -9% | 0.2463 | -| TPC-H 17 || 514.3 | 603.7 | +17% || 0.47 | 0.43 | -9% | 0.1988 | -| TPC-H 18 || 3411.6 | 3622.1 | +6% || 0.47 | 0.42 | -9% | 0.0151 | -| TPC-H 19 || 728.6 | 813.4 | +12% || 0.47 | 0.43 | -9% | 0.4628 | -| TPC-H 20 || 1278.3 | 1325.4 | +4% || 0.47 | 0.43 | -9% | 0.7164 | -| TPC-H 21 || 5188.8 | 6078.0 | +17% || 0.47 | 0.42 | -9% | 0.0006 | +| TPC-H 22 || 1048.9 | 912.6 | -13% || 0.47 | 0.43 | -9% | 0.1856 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 55624.3 | 60800.5 | +9% || | | | | -| Geomean || | | || | | -9% | | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - single-threaded

Sum of avg. item runtimes: -2% || Geometric mean of throughput changes: +2%

Configuration Overview - click to expand

```diff +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 09:29:00 | 2022-07-23 12:16:33 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 259.2 | 262.0 | +1%˄ || 3.86 | 3.82 | -1%˄ | 0.0144 | | 03 || 69.0 | 67.3 | -2%˄ || 14.49 | 14.86 | +3%˄ | 0.0000 | | 06 || 152.6 | 149.3 | -2%˄ || 6.55 | 6.70 | +2%˄ | 0.0000 | +| 07 || 296.9 | 263.9 | -11%˄ || 3.37 | 3.79 | +12%˄ | 0.0000 | | 09 || 634.0 | 637.8 | +1% || 1.58 | 1.57 | -1% | 0.0000 | +| 10 || 129.2 | 119.6 | -7%˄ || 7.74 | 8.36 | +8%˄ | 0.0000 | +| 13 || 426.0 | 396.7 | -7%˄ || 2.35 | 2.52 | +7%˄ | 0.0000 | | 15 || 103.4 | 106.4 | +3%˄ || 9.67 | 9.40 | -3%˄ | 0.0000 | +| 16 || 152.3 | 139.7 | -8%˄ || 6.57 | 7.16 | +9%˄ | 0.0000 | | 17 || 308.4 | 302.6 | -2%˄ || 3.24 | 3.30 | +2%˄ | 0.0000 | | 19 || 100.0 | 96.0 | -4%˄ || 10.00 | 10.42 | +4%˄ | 0.0000 | | 25 || 162.8 | 158.9 | -2%˄ || 6.14 | 6.29 | +2%˄ | 0.0000 | +| 26 || 134.0 | 125.5 | -6%˄ || 7.46 | 7.97 | +7%˄ | 0.0000 | | 28 || 608.9 | 597.8 | -2%˄ || 1.64 | 1.67 | +2%˄ | 0.0000 | | 29 || 469.4 | 467.2 | -0%˄ || 2.13 | 2.14 | +0%˄ | 0.0002 | | 31 || 1268.3 | 1256.0 | -1% || 0.79 | 0.80 | +1% | 0.0003 | -| 32 || 40.5 | 42.4 | +5%˄ || 24.68 | 23.57 | -5%˄ | 0.0000 | +| 34 || 159.0 | 147.2 | -7%˄ || 6.29 | 6.79 | +8%˄ | 0.0000 | | 35 || 605.1 | 628.3 | +4%˄ || 1.65 | 1.59 | -4%˄ | 0.0000 | +| 37 || 275.9 | 234.1 | -15%˄ || 3.62 | 4.27 | +18%˄ | 0.0000 | +| 39a || 1736.9 | 1655.4 | -5% || 0.58 | 0.60 | +5% | 0.0000 | | 39b || 1728.0 | 1655.5 | -4% || 0.58 | 0.60 | +4% | 0.0000 | | 41 || 288.5 | 289.1 | +0%˄ || 3.47 | 3.46 | -0%˄ | 0.1292 | +| 42 || 80.9 | 77.3 | -4%˄ || 12.36 | 12.94 | +5%˄ | 0.0000 | | 43 || 942.7 | 911.6 | -3% || 1.06 | 1.10 | +3% | 0.0000 | | 45 || 114.2 | 117.3 | +3%˄ || 8.76 | 8.53 | -3%˄ | 0.0000 | | 48 || 1014.0 | 988.0 | -3% || 0.99 | 1.01 | +3% | 0.0000 | | 50 || 109.3 | 109.5 | +0%˄ || 9.15 | 9.13 | -0%˄ | 0.5515 | | 52 || 78.9 | 78.0 | -1%˄ || 12.67 | 12.81 | +1%˄ | 0.0000 | | 55 || 75.4 | 74.8 | -1%˄ || 13.26 | 13.37 | +1%˄ | 0.0000 | | 62 || 544.3 | 524.1 | -4%˄ || 1.84 | 1.91 | +4%˄ | 0.0000 | | 65 || 1763.4 | 1718.7 | -3% || 0.57 | 0.58 | +3% | 0.0000 | | 69 || 119.5 | 119.3 | -0%˄ || 8.37 | 8.38 | +0%˄ | 0.4041 | | 73 || 76.6 | 75.4 | -2%˄ || 13.05 | 13.26 | +2%˄ | 0.0000 | | 79 || 467.1 | 463.9 | -1%˄ || 2.14 | 2.16 | +1%˄ | 0.0000 | | 81 || 209.4 | 202.2 | -3%˄ || 4.78 | 4.95 | +4%˄ | 0.0000 | +| 82 || 351.1 | 328.8 | -6%˄ || 2.85 | 3.04 | +7%˄ | 0.0000 | | 83 || 39.8 | 39.4 | -1%˄ || 25.15 | 25.35 | +1%˄ | 0.3982 | +| 85 || 134.0 | 124.7 | -7%˄ || 7.46 | 8.02 | +7%˄ | 0.0000 | -| 88 || 617.1 | 657.2 | +7% || 1.62 | 1.52 | -6% | 0.0000 | -| 91 || 15.4 | 16.2 | +6%˄ || 65.08 | 61.56 | -5%˄ | 0.0000 | | 92 || 34.7 | 35.8 | +3%˄ || 28.79 | 27.95 | -3%˄ | 0.0000 | | 93 || 3878.5 | 3827.5 | -1% || 0.26 | 0.26 | +1% | 0.0000 | | 94 || 97.9 | 95.0 | -3%˄ || 10.22 | 10.53 | +3%˄ | 0.0000 | | 95 || 9009.6 | 8951.6 | -1% || 0.11 | 0.11 | +1% | ˅ | | 96 || 57.3 | 57.7 | +1%˄ || 17.44 | 17.33 | -1%˄ | 0.5730 | | 97 || 3242.2 | 3269.7 | +1% || 0.31 | 0.31 | -1% | 0.0790 | | 99 || 1055.5 | 1019.3 | -3% || 0.95 | 0.98 | +4% | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 34237.3 | 33681.5 | -2% || | | | | | Geomean || | | || | | +2% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | | || ˅ Insufficient number of runs for p-value calculation | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 09:55:01 | 2022-07-23 12:44:27 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 629.4 | 653.1 | +4% || 0.47 | 0.47 | -0% | 0.7324 | +| 03 || 234.2 | 185.4 | -21% || 0.47 | 0.47 | -0% | 0.3023 | | 06 || 665.7 | 654.4 | -2% || 0.47 | 0.47 | -0% | 0.8877 | -| 07 || 760.8 | 843.3 | +11% || 0.47 | 0.47 | -0% | 0.3291 | | 09 || 1295.6 | 1286.6 | -1% || 0.47 | 0.47 | -0% | 0.9020 | -| 10 || 421.8 | 463.7 | +10% || 0.47 | 0.47 | -0% | 0.5381 | | 13 || 1504.7 | 1506.3 | +0% || 0.47 | 0.47 | -0% | 0.9905 | -| 15 || 380.0 | 461.6 | +21% || 0.47 | 0.47 | -0% | 0.2312 | +| 16 || 824.3 | 753.5 | -9% || 0.47 | 0.47 | -0% | 0.4938 | | 17 || 1015.2 | 1001.2 | -1% || 0.47 | 0.47 | -0% | 0.8891 | -| 19 || 383.0 | 492.0 | +28% || 0.47 | 0.47 | -0% | 0.0542 | +| 25 || 824.4 | 742.4 | -10% || 0.47 | 0.47 | -0% | 0.4174 | | 26 || 507.5 | 527.6 | +4% || 0.47 | 0.47 | -0% | 0.7825 | | 28 || 560.7 | 537.2 | -4% || 0.47 | 0.47 | -0% | 0.7336 | | 29 || 1329.2 | 1305.0 | -2% || 0.47 | 0.47 | -0% | 0.8256 | -| 31 || 1849.9 | 2008.8 | +9% || 0.47 | 0.47 | -0% | 0.2894 | -| 32 || 234.5 | 290.7 | +24% || 0.47 | 0.47 | -0% | 0.2564 | | 34 || 421.0 | 417.2 | -1% || 0.47 | 0.47 | -0% | 0.9272 | | 35 || 1951.9 | 2024.7 | +4% || 0.47 | 0.47 | +0% | 0.6448 | | 37 || 565.9 | 589.3 | +4% || 0.47 | 0.47 | -0% | 0.7635 | -| 39a || 2029.8 | 2206.6 | +9% || 0.47 | 0.47 | -0% | 0.1281 | | 39b || 2123.8 | 2132.4 | +0% || 0.47 | 0.47 | -0% | 0.9364 | | 41 || 2704.4 | 2760.2 | +2% || 0.47 | 0.47 | -0% | 0.6951 | | 42 || 322.3 | 331.1 | +3% || 0.47 | 0.47 | -0% | 0.8852 | | 43 || 1350.8 | 1378.3 | +2% || 0.47 | 0.47 | -0% | 0.7780 | +| 45 || 636.9 | 506.1 | -21% || 0.47 | 0.47 | -0% | 0.1258 | -| 48 || 2534.6 | 2699.3 | +6% || 0.47 | 0.47 | -0% | 0.3111 | | 50 || 495.5 | 504.3 | +2% || 0.47 | 0.47 | -0% | 0.8819 | +| 52 || 313.5 | 293.9 | -6% || 0.47 | 0.47 | -0% | 0.6872 | | 55 || 286.8 | 275.5 | -4% || 0.47 | 0.47 | -0% | 0.8531 | +| 62 || 838.8 | 789.9 | -6% || 0.47 | 0.47 | -0% | 0.4540 | | 65 || 3623.5 | 3621.8 | -0% || 0.47 | 0.47 | -0% | 0.9837 | -| 69 || 597.2 | 627.2 | +5% || 0.47 | 0.47 | -0% | 0.7428 | +| 73 || 307.7 | 273.7 | -11% || 0.47 | 0.47 | -0% | 0.5189 | | 79 || 1200.3 | 1183.9 | -1% || 0.47 | 0.47 | -0% | 0.8571 | -| 81 || 728.2 | 880.8 | +21% || 0.47 | 0.47 | -0% | 0.0782 | | 82 || 720.3 | 705.2 | -2% || 0.47 | 0.47 | -0% | 0.8511 | +| 83 || 309.6 | 252.3 | -19% || 0.47 | 0.47 | -0% | 0.3586 | +| 85 || 744.1 | 692.4 | -7% || 0.47 | 0.47 | -0% | 0.6086 | -| 88 || 655.8 | 837.6 | +28% || 0.47 | 0.47 | -0% | 0.0308 | +| 91 || 194.1 | 155.5 | -20% || 0.47 | 0.47 | -0% | 0.3959 | +| 92 || 339.8 | 245.4 | -28% || 0.47 | 0.47 | -0% | 0.1164 | | 93 || 2302.2 | 2333.9 | +1% || 0.47 | 0.47 | -0% | 0.8055 | -| 94 || 631.6 | 688.4 | +9% || 0.47 | 0.47 | -0% | 0.5998 | | 95 || 7935.6 | 7781.7 | -2% || 0.47 | 0.47 | -0% | 0.3475 | +| 96 || 261.5 | 171.3 | -35% || 0.47 | 0.47 | -0% | 0.0379 | | 97 || 4287.2 | 4351.0 | +1% || 0.47 | 0.47 | -0% | 0.6654 | +| 99 || 1399.9 | 1227.9 | -12% || 0.47 | 0.47 | -0% | 0.0282 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 56235.4 | 56651.4 | +1% || | | | | | Geomean || | | || | | -0% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - single-threaded

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 10:15:34 | 2022-07-23 13:04:57 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 22.9 | 22.8 | -0% || 4.25 | 4.28 | +1% | 0.7454 | | New-Order || 16.5 | 16.3 | -1% || 47.86 | 48.19 | +1% | 0.2188 | | Order-Status || 1.1 | 1.1 | -0% || 4.26 | 4.30 | +1% | 0.9607 | | Payment || 2.0 | 2.1 | +1% || 45.73 | 46.03 | +1% | 0.0171 | | Stock-Level || 3.5 | 3.5 | +0% || 4.25 | 4.27 | +0% | 0.7389 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 46.0 | 45.8 | -0% || | | | | | Geomean || | | || | | +1% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 10:16:35 | 2022-07-23 13:05:59 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 219.1 | 219.8 | +0% || 4.32 | 4.31 | -0% | 0.6636 | | unsucc.: || 4.3 | 4.3 | -1% || 62.88 | 62.94 | +0% | | | New-Order || 57.3 | 57.2 | -0% || 134.12 | 133.68 | -0% | 0.3948 | | unsucc.: || 4.9 | 5.0 | +1% || 621.96 | 622.86 | +0% | | | Order-Status || 7.6 | 7.6 | -1% || 67.21 | 67.25 | +0% | 0.0700 | | Payment || 9.8 | 9.9 | +0% || 21.57 | 21.50 | -0% | 0.6081 | | unsucc.: || 4.5 | 4.4 | -0% || 700.91 | 701.43 | +0% | | | Stock-Level || 17.3 | 17.3 | +0% || 67.21 | 67.25 | +0% | 0.7358 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 311.2 | 311.8 | +0% || | | | | | Geomean || | | || | | -0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - single-threaded

Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -1%

Configuration Overview - click to expand

```diff +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 10:36:42 | 2022-07-23 13:27:39 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 10a || 166.8 | 164.0 | -2%˄ || 5.99 | 6.10 | +2%˄ | 0.0000 | | 10b || 116.4 | 115.3 | -1%˄ || 8.59 | 8.67 | +1%˄ | 0.0001 | -| 10c || 322.2 | 351.6 | +9%˄ || 3.10 | 2.84 | -8%˄ | 0.0000 | | 11a || 15.2 | 15.2 | -0%˄ || 65.67 | 65.78 | +0%˄ | 0.0009 | | 11b || 14.9 | 15.1 | +1%˄ || 67.10 | 66.19 | -1%˄ | 0.0000 | | 11c || 20.6 | 20.4 | -1%˄ || 48.52 | 48.90 | +1%˄ | 0.0001 | | 11d || 23.4 | 23.2 | -1%˄ || 42.65 | 43.17 | +1%˄ | 0.0000 | | 12a || 48.4 | 47.3 | -2%˄ || 20.66 | 21.16 | +2%˄ | 0.0000 | | 12b || 29.2 | 29.3 | +0%˄ || 34.27 | 34.12 | -0%˄ | 0.0000 | | 12c || 83.0 | 84.8 | +2%˄ || 12.05 | 11.79 | -2%˄ | 0.0000 | | 13a || 274.8 | 271.9 | -1%˄ || 3.64 | 3.68 | +1%˄ | 0.0000 | | 13b || 152.1 | 154.4 | +1%˄ || 6.57 | 6.48 | -1%˄ | 0.0000 | | 13c || 115.3 | 118.7 | +3%˄ || 8.67 | 8.42 | -3%˄ | 0.0000 | +| 13d || 586.7 | 558.8 | -5%˄ || 1.70 | 1.79 | +5%˄ | 0.0000 | | 14a || 415.8 | 421.3 | +1%˄ || 2.41 | 2.37 | -1%˄ | 0.0000 | -| 14b || 232.8 | 249.0 | +7%˄ || 4.29 | 4.02 | -6%˄ | 0.0000 | | 14c || 493.0 | 496.5 | +1%˄ || 2.03 | 2.01 | -1%˄ | 0.0000 | | 15a || 81.2 | 83.5 | +3%˄ || 12.32 | 11.97 | -3%˄ | 0.0000 | | 15b || 80.5 | 82.9 | +3%˄ || 12.41 | 12.06 | -3%˄ | 0.0000 | | 15c || 85.6 | 88.2 | +3%˄ || 11.68 | 11.34 | -3%˄ | 0.0000 | | 15d || 82.1 | 85.0 | +4%˄ || 12.18 | 11.76 | -3%˄ | 0.0000 | | 16a || 2226.5 | 2310.9 | +4% || 0.45 | 0.43 | -4% | 0.0000 | | 16b || 3273.4 | 3336.6 | +2% || 0.31 | 0.30 | -2% | 0.0012 | | 16c || 2328.2 | 2405.2 | +3% || 0.43 | 0.42 | -3% | 0.0000 | | 16d || 2298.1 | 2378.8 | +4% || 0.44 | 0.42 | -3% | 0.0000 | | 17a || 580.6 | 601.3 | +4%˄ || 1.72 | 1.66 | -3%˄ | 0.0000 | | 17b || 457.9 | 477.0 | +4%˄ || 2.18 | 2.10 | -4%˄ | 0.0000 | | 17c || 436.9 | 455.6 | +4%˄ || 2.29 | 2.19 | -4%˄ | 0.0000 | | 17d || 516.8 | 536.4 | +4%˄ || 1.93 | 1.86 | -4%˄ | 0.0000 | | 17e || 1582.0 | 1599.2 | +1% || 0.63 | 0.63 | -1% | 0.0000 | | 17f || 938.5 | 965.0 | +3% || 1.07 | 1.04 | -3% | 0.0000 | -| 18a || 598.0 | 636.2 | +6%˄ || 1.67 | 1.57 | -6%˄ | 0.0000 | | 18b || 112.3 | 110.7 | -1%˄ || 8.91 | 9.03 | +1%˄ | 0.0000 | | 18c || 557.9 | 569.5 | +2%˄ || 1.79 | 1.76 | -2%˄ | 0.0000 | | 19a || 180.7 | 185.6 | +3%˄ || 5.53 | 5.39 | -3%˄ | 0.0000 | | 19b || 124.3 | 126.3 | +2%˄ || 8.04 | 7.92 | -2%˄ | 0.0000 | | 19c || 192.6 | 197.8 | +3%˄ || 5.19 | 5.05 | -3%˄ | 0.0000 | | 19d || 717.3 | 727.7 | +1% || 1.39 | 1.37 | -1% | 0.0000 | | 1a || 9.0 | 9.0 | -1%˄ || 110.59 | 111.44 | +1%˄ | 0.0000 | | 1b || 8.5 | 8.4 | -1%˄ || 118.10 | 118.96 | +1%˄ | 0.0009 | | 1c || 9.5 | 9.5 | +0%˄ || 104.99 | 104.72 | -0%˄ | 0.0004 | | 1d || 8.5 | 8.4 | -0%˄ || 118.24 | 118.36 | +0%˄ | 0.6758 | | 20a || 390.6 | 387.4 | -1%˄ || 2.56 | 2.58 | +1%˄ | 0.0000 | | 20b || 228.2 | 226.6 | -1%˄ || 4.38 | 4.41 | +1%˄ | 0.0000 | | 20c || 219.8 | 218.7 | -1%˄ || 4.55 | 4.57 | +1%˄ | 0.0000 | | 21a || 35.1 | 35.4 | +1%˄ || 28.47 | 28.23 | -1%˄ | 0.0000 | | 21b || 19.1 | 19.2 | +1%˄ || 52.48 | 52.10 | -1%˄ | 0.0000 | | 21c || 34.7 | 35.0 | +1%˄ || 28.86 | 28.59 | -1%˄ | 0.0000 | -| 22a || 195.6 | 209.4 | +7%˄ || 5.11 | 4.77 | -7%˄ | 0.0000 | -| 22b || 142.5 | 149.8 | +5%˄ || 7.02 | 6.67 | -5%˄ | 0.0000 | | 22c || 493.0 | 507.1 | +3%˄ || 2.03 | 1.97 | -3%˄ | 0.0000 | | 22d || 895.6 | 912.7 | +2% || 1.12 | 1.10 | -2% | 0.0000 | | 23a || 42.8 | 42.8 | +0%˄ || 23.39 | 23.38 | -0%˄ | 0.9481 | | 23b || 43.9 | 44.0 | +0%˄ || 22.76 | 22.75 | -0%˄ | 0.4999 | | 23c || 85.5 | 84.6 | -1%˄ || 11.69 | 11.82 | +1%˄ | 0.0000 | | 24a || 144.1 | 144.0 | -0%˄ || 6.94 | 6.95 | +0%˄ | 0.3636 | | 24b || 122.8 | 123.1 | +0%˄ || 8.14 | 8.12 | -0%˄ | 0.0507 | | 25a || 276.5 | 279.2 | +1%˄ || 3.62 | 3.58 | -1%˄ | 0.0000 | | 25b || 113.3 | 113.1 | -0%˄ || 8.82 | 8.84 | +0%˄ | 0.0232 | | 25c || 908.2 | 922.4 | +2% || 1.10 | 1.08 | -2% | 0.0000 | | 26a || 163.5 | 160.7 | -2%˄ || 6.12 | 6.22 | +2%˄ | 0.0000 | | 26b || 131.0 | 129.2 | -1%˄ || 7.63 | 7.74 | +1%˄ | 0.0000 | | 26c || 232.5 | 230.6 | -1%˄ || 4.30 | 4.34 | +1%˄ | 0.0000 | | 27a || 22.6 | 23.0 | +2%˄ || 44.28 | 43.49 | -2%˄ | 0.0000 | | 27b || 22.2 | 22.5 | +1%˄ || 44.98 | 44.52 | -1%˄ | 0.0000 | | 27c || 34.7 | 35.1 | +1%˄ || 28.81 | 28.45 | -1%˄ | 0.0000 | | 28a || 290.0 | 298.9 | +3%˄ || 3.45 | 3.35 | -3%˄ | 0.0000 | -| 28b || 37.2 | 49.5 | +33%˄ || 26.90 | 20.18 | -25%˄ | 0.0000 | -| 28c || 328.1 | 343.8 | +5%˄ || 3.05 | 2.91 | -5%˄ | 0.0000 | | 29a || 101.8 | 102.4 | +1%˄ || 9.82 | 9.76 | -1%˄ | 0.0000 | | 29b || 76.2 | 76.8 | +1%˄ || 13.12 | 13.03 | -1%˄ | 0.0000 | | 29c || 148.5 | 148.3 | -0%˄ || 6.73 | 6.74 | +0%˄ | 0.0274 | | 2a || 46.1 | 46.7 | +1%˄ || 21.67 | 21.42 | -1%˄ | 0.0000 | | 2b || 37.4 | 38.3 | +2%˄ || 26.70 | 26.11 | -2%˄ | 0.0000 | | 2c || 21.8 | 22.3 | +2%˄ || 45.93 | 44.88 | -2%˄ | 0.0000 | | 2d || 64.6 | 66.8 | +3%˄ || 15.47 | 14.98 | -3%˄ | 0.0000 | | 30a || 148.6 | 148.5 | -0%˄ || 6.73 | 6.73 | +0%˄ | 0.1076 | | 30b || 127.5 | 126.4 | -1%˄ || 7.84 | 7.91 | +1%˄ | 0.0000 | | 30c || 377.1 | 386.7 | +3%˄ || 2.65 | 2.59 | -2%˄ | 0.0000 | | 31a || 146.0 | 146.0 | -0%˄ || 6.85 | 6.85 | +0%˄ | 0.8171 | | 31b || 124.6 | 124.4 | -0%˄ || 8.03 | 8.04 | +0%˄ | 0.0170 | | 31c || 170.4 | 168.6 | -1%˄ || 5.87 | 5.93 | +1%˄ | 0.0000 | | 32a || 12.9 | 12.9 | -0%˄ || 77.21 | 77.26 | +0%˄ | 0.0901 | -| 32b || 30.9 | 32.9 | +6%˄ || 32.33 | 30.42 | -6%˄ | 0.0000 | | 33a || 19.5 | 19.4 | -0%˄ || 51.34 | 51.51 | +0%˄ | 0.1104 | | 33b || 18.9 | 18.7 | -1%˄ || 52.94 | 53.43 | +1%˄ | 0.0000 | | 33c || 22.8 | 22.6 | -1%˄ || 43.94 | 44.31 | +1%˄ | 0.0000 | | 3a || 122.7 | 122.9 | +0%˄ || 8.15 | 8.14 | -0%˄ | 0.4129 | | 3b || 12.7 | 12.6 | -0%˄ || 78.91 | 79.25 | +0%˄ | 0.0000 | | 3c || 436.1 | 449.8 | +3%˄ || 2.29 | 2.22 | -3%˄ | 0.0000 | +| 4a || 112.5 | 104.4 | -7%˄ || 8.89 | 9.58 | +8%˄ | 0.0000 | | 4b || 11.5 | 11.4 | -1%˄ || 86.71 | 87.38 | +1%˄ | 0.0105 | | 4c || 124.0 | 120.6 | -3%˄ || 8.07 | 8.29 | +3%˄ | 0.0000 | | 5a || 52.5 | 53.7 | +2%˄ || 19.04 | 18.62 | -2%˄ | 0.0000 | | 5b || 134.8 | 134.4 | -0%˄ || 7.42 | 7.44 | +0%˄ | 0.0832 | -| 5c || 185.1 | 198.2 | +7%˄ || 5.40 | 5.04 | -7%˄ | 0.0000 | | 6a || 103.5 | 102.5 | -1%˄ || 9.66 | 9.75 | +1%˄ | 0.0000 | | 6b || 115.5 | 114.4 | -1%˄ || 8.66 | 8.74 | +1%˄ | 0.0000 | | 6c || 99.0 | 98.1 | -1%˄ || 10.10 | 10.19 | +1%˄ | 0.0000 | | 6d || 527.4 | 535.1 | +1%˄ || 1.90 | 1.87 | -1%˄ | 0.0000 | | 6e || 103.5 | 102.7 | -1%˄ || 9.66 | 9.74 | +1%˄ | 0.0000 | | 6f || 696.7 | 706.2 | +1% || 1.44 | 1.42 | -1% | 0.0000 | | 7a || 53.3 | 53.8 | +1%˄ || 18.77 | 18.60 | -1%˄ | 0.0000 | | 7b || 49.6 | 50.1 | +1%˄ || 20.17 | 19.96 | -1%˄ | 0.0000 | | 7c || 582.6 | 565.2 | -3%˄ || 1.72 | 1.77 | +3%˄ | 0.0000 | | 8a || 38.0 | 39.1 | +3%˄ || 26.30 | 25.58 | -3%˄ | 0.0000 | | 8b || 35.8 | 37.2 | +4%˄ || 27.95 | 26.91 | -4%˄ | 0.0000 | | 8c || 1858.3 | 1940.3 | +4% || 0.54 | 0.52 | -4% | 0.0000 | -| 8d || 303.8 | 328.6 | +8%˄ || 3.29 | 3.04 | -8%˄ | 0.0000 | | 9a || 187.1 | 193.3 | +3%˄ || 5.35 | 5.17 | -3%˄ | 0.0000 | | 9b || 104.5 | 106.6 | +2%˄ || 9.57 | 9.38 | -2%˄ | 0.0000 | | 9c || 269.1 | 274.1 | +2%˄ || 3.72 | 3.65 | -2%˄ | 0.0000 | | 9d || 471.9 | 481.6 | +2%˄ || 2.12 | 2.08 | -2%˄ | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 34738.0 | 35525.0 | +2% || | | | | | Geomean || | | || | | -1% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +5% || Geometric mean of throughput changes: -5%

Configuration Overview - click to expand

```diff +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 11:18:48 | 2022-07-23 14:10:35 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ -| 10a || 190.7 | 196.0 | +3% || 0.51 | 0.49 | -5% | 0.8222 | | 10b || 128.6 | 128.4 | -0% || 0.51 | 0.49 | -4% | 0.9927 | +| 10c || 439.8 | 419.9 | -5% || 0.51 | 0.49 | -4% | 0.5882 | +| 11a || 58.6 | 43.6 | -26% || 0.51 | 0.49 | -5% | 0.2633 | -| 11b || 47.7 | 45.7 | -4% || 0.51 | 0.49 | -5% | 0.8510 | +| 11c || 73.5 | 57.8 | -21% || 0.51 | 0.49 | -5% | 0.2558 | -| 11d || 64.6 | 84.4 | +31% || 0.51 | 0.49 | -5% | 0.1240 | +| 12a || 190.1 | 168.9 | -11% || 0.51 | 0.49 | -5% | 0.3386 | -| 12b || 60.1 | 64.9 | +8% || 0.51 | 0.49 | -5% | 0.6857 | -| 12c || 282.4 | 299.1 | +6% || 0.51 | 0.49 | -5% | 0.5400 | -| 13a || 636.9 | 665.4 | +4% || 0.51 | 0.49 | -5% | 0.5550 | -| 13b || 231.4 | 310.6 | +34% || 0.51 | 0.49 | -5% | 0.0100 | +| 13c || 273.0 | 207.0 | -24% || 0.51 | 0.49 | -5% | 0.0249 | -| 13d || 1045.9 | 1010.5 | -3% || 0.51 | 0.49 | -5% | 0.5086 | -| 14a || 802.0 | 849.7 | +6% || 0.51 | 0.49 | -4% | 0.3372 | -| 14b || 442.7 | 467.3 | +6% || 0.51 | 0.49 | -5% | 0.4965 | -| 14c || 901.2 | 909.7 | +1% || 0.51 | 0.49 | -5% | 0.8544 | -| 15a || 139.7 | 163.4 | +17% || 0.51 | 0.49 | -5% | 0.2273 | -| 15b || 117.7 | 144.7 | +23% || 0.51 | 0.49 | -5% | 0.1622 | -| 15c || 151.5 | 150.7 | -1% || 0.51 | 0.49 | -5% | 0.9683 | +| 15d || 230.5 | 210.1 | -9% || 0.51 | 0.49 | -5% | 0.5731 | -| 16a || 2499.7 | 2611.4 | +4% || 0.51 | 0.49 | -5% | 0.0649 | -| 16b || 3777.2 | 4013.7 | +6% || 0.51 | 0.48 | -5% | 0.0004 | -| 16c || 2579.8 | 2647.6 | +3% || 0.51 | 0.48 | -5% | 0.1620 | -| 16d || 2608.6 | 2736.3 | +5% || 0.51 | 0.49 | -5% | 0.0233 | +| 17a || 854.0 | 753.2 | -12% || 0.51 | 0.49 | -5% | 0.0667 | -| 17b || 564.4 | 647.4 | +15% || 0.51 | 0.49 | -5% | 0.0555 | -| 17c || 506.1 | 581.2 | +15% || 0.51 | 0.49 | -5% | 0.0830 | -| 17d || 570.8 | 575.7 | +1% || 0.51 | 0.49 | -5% | 0.9064 | -| 17e || 1488.6 | 1590.1 | +7% || 0.51 | 0.48 | -5% | 0.0363 | -| 17f || 1009.2 | 1151.5 | +14% || 0.51 | 0.49 | -5% | 0.0038 | -| 18a || 729.9 | 749.0 | +3% || 0.51 | 0.49 | -5% | 0.6634 | -| 18b || 139.6 | 166.3 | +19% || 0.51 | 0.49 | -5% | 0.1538 | -| 18c || 701.2 | 676.0 | -4% || 0.51 | 0.49 | -5% | 0.5666 | -| 19a || 498.1 | 505.3 | +1% || 0.51 | 0.49 | -5% | 0.8863 | -| 19b || 351.6 | 378.4 | +8% || 0.51 | 0.49 | -5% | 0.5155 | -| 19c || 480.6 | 549.6 | +14% || 0.51 | 0.49 | -5% | 0.1295 | -| 19d || 1201.1 | 1252.7 | +4% || 0.51 | 0.49 | -5% | 0.3398 | -| 1a || 31.2 | 32.1 | +3% || 0.51 | 0.49 | -5% | 0.9091 | -| 1b || 23.3 | 36.9 | +59% || 0.51 | 0.49 | -5% | 0.0738 | -| 1c || 36.7 | 46.9 | +28% || 0.51 | 0.49 | -5% | 0.3514 | +| 1d || 40.2 | 28.8 | -28% || 0.51 | 0.49 | -5% | 0.3481 | -| 20a || 631.2 | 608.8 | -4% || 0.51 | 0.49 | -5% | 0.6546 | -| 20b || 387.0 | 409.2 | +6% || 0.51 | 0.49 | -5% | 0.5058 | +| 20c || 435.8 | 406.8 | -7% || 0.51 | 0.49 | -5% | 0.4314 | -| 21a || 89.7 | 96.2 | +7% || 0.51 | 0.49 | -5% | 0.7717 | -| 21b || 61.2 | 72.3 | +18% || 0.51 | 0.49 | -5% | 0.3334 | -| 21c || 55.2 | 73.5 | +33% || 0.51 | 0.49 | -4% | 0.0574 | -| 22a || 523.3 | 504.8 | -4% || 0.51 | 0.49 | -5% | 0.6532 | -| 22b || 318.9 | 366.1 | +15% || 0.51 | 0.49 | -5% | 0.1773 | -| 22c || 805.6 | 911.8 | +13% || 0.51 | 0.49 | -5% | 0.0250 | -| 22d || 1191.3 | 1344.9 | +13% || 0.51 | 0.49 | -5% | 0.0164 | +| 23a || 152.4 | 127.7 | -16% || 0.51 | 0.49 | -5% | 0.2197 | +| 23b || 126.0 | 116.5 | -8% || 0.51 | 0.49 | -4% | 0.6618 | +| 23c || 196.3 | 172.9 | -12% || 0.51 | 0.49 | -5% | 0.4062 | -| 24a || 236.6 | 250.2 | +6% || 0.51 | 0.49 | -5% | 0.6573 | -| 24b || 151.0 | 157.1 | +4% || 0.51 | 0.49 | -5% | 0.7754 | -| 25a || 535.9 | 548.2 | +2% || 0.51 | 0.49 | -5% | 0.7941 | -| 25b || 128.2 | 128.2 | -0% || 0.51 | 0.49 | -5% | 0.9994 | -| 25c || 946.6 | 1038.5 | +10% || 0.51 | 0.49 | -5% | 0.1394 | -| 26a || 319.4 | 315.6 | -1% || 0.51 | 0.49 | -5% | 0.9062 | -| 26b || 199.1 | 208.1 | +5% || 0.51 | 0.49 | -5% | 0.6705 | +| 26c || 503.6 | 475.2 | -6% || 0.51 | 0.49 | -5% | 0.5406 | -| 27a || 95.9 | 109.8 | +15% || 0.51 | 0.49 | -5% | 0.4816 | -| 27b || 79.4 | 104.0 | +31% || 0.51 | 0.49 | -5% | 0.1905 | +| 27c || 91.0 | 80.2 | -12% || 0.51 | 0.49 | -5% | 0.5566 | -| 28a || 692.8 | 775.2 | +12% || 0.51 | 0.49 | -5% | 0.1113 | -| 28b || 169.6 | 195.7 | +15% || 0.51 | 0.49 | -5% | 0.2984 | -| 28c || 690.2 | 758.6 | +10% || 0.51 | 0.49 | -5% | 0.1295 | +| 29a || 242.9 | 206.5 | -15% || 0.51 | 0.49 | -5% | 0.2817 | -| 29b || 130.7 | 147.2 | +13% || 0.51 | 0.49 | -5% | 0.3992 | -| 29c || 224.4 | 237.3 | +6% || 0.51 | 0.49 | -5% | 0.6895 | +| 2a || 198.7 | 172.5 | -13% || 0.51 | 0.49 | -5% | 0.3709 | -| 2b || 123.4 | 140.0 | +13% || 0.51 | 0.49 | -5% | 0.4039 | -| 2c || 80.5 | 92.9 | +15% || 0.51 | 0.49 | -5% | 0.4757 | -| 2d || 196.4 | 243.3 | +24% || 0.51 | 0.49 | -5% | 0.1336 | -| 30a || 265.3 | 326.1 | +23% || 0.51 | 0.49 | -5% | 0.0299 | -| 30b || 197.8 | 233.0 | +18% || 0.51 | 0.49 | -4% | 0.1183 | -| 30c || 709.7 | 727.8 | +3% || 0.51 | 0.49 | -5% | 0.7326 | -| 31a || 308.6 | 340.0 | +10% || 0.51 | 0.49 | -5% | 0.4235 | +| 31b || 171.9 | 159.3 | -7% || 0.51 | 0.49 | -4% | 0.5901 | -| 31c || 310.5 | 337.6 | +9% || 0.51 | 0.49 | -5% | 0.4090 | +| 32a || 48.1 | 39.4 | -18% || 0.51 | 0.49 | -5% | 0.4528 | +| 32b || 140.4 | 130.9 | -7% || 0.51 | 0.49 | -5% | 0.6670 | -| 33a || 58.5 | 60.0 | +3% || 0.51 | 0.49 | -5% | 0.8929 | +| 33b || 71.3 | 56.8 | -20% || 0.51 | 0.49 | -5% | 0.3240 | +| 33c || 102.3 | 68.3 | -33% || 0.51 | 0.49 | -5% | 0.0405 | -| 3a || 349.9 | 398.9 | +14% || 0.51 | 0.49 | -5% | 0.2693 | -| 3b || 52.9 | 83.1 | +57% || 0.51 | 0.49 | -5% | 0.1247 | +| 3c || 530.8 | 505.8 | -5% || 0.51 | 0.49 | -5% | 0.4684 | -| 4a || 210.0 | 211.8 | +1% || 0.51 | 0.49 | -5% | 0.9497 | +| 4b || 51.9 | 45.7 | -12% || 0.51 | 0.49 | -5% | 0.6929 | -| 4c || 205.0 | 214.9 | +5% || 0.51 | 0.49 | -4% | 0.6686 | +| 5a || 180.1 | 161.0 | -11% || 0.51 | 0.49 | -5% | 0.4761 | -| 5b || 182.8 | 185.9 | +2% || 0.51 | 0.49 | -5% | 0.8828 | -| 5c || 212.3 | 228.2 | +8% || 0.51 | 0.49 | -5% | 0.5481 | -| 6a || 88.0 | 106.1 | +21% || 0.51 | 0.49 | -5% | 0.2518 | +| 6b || 184.0 | 125.5 | -32% || 0.51 | 0.49 | -5% | 0.0127 | -| 6c || 102.1 | 101.8 | -0% || 0.51 | 0.49 | -5% | 0.9875 | -| 6d || 507.3 | 521.3 | +3% || 0.51 | 0.49 | -5% | 0.6411 | -| 6e || 89.3 | 130.0 | +46% || 0.51 | 0.49 | -4% | 0.0704 | -| 6f || 969.2 | 946.2 | -2% || 0.51 | 0.49 | -5% | 0.5562 | -| 7a || 103.7 | 108.9 | +5% || 0.51 | 0.49 | -5% | 0.7357 | -| 7b || 95.8 | 137.2 | +43% || 0.51 | 0.49 | -5% | 0.0481 | -| 7c || 798.9 | 832.5 | +4% || 0.51 | 0.49 | -5% | 0.4283 | -| 8a || 127.2 | 150.0 | +18% || 0.51 | 0.49 | -5% | 0.2884 | -| 8b || 122.0 | 133.4 | +9% || 0.51 | 0.49 | -5% | 0.5862 | -| 8c || 2107.0 | 2382.9 | +13% || 0.51 | 0.49 | -5% | 0.0000 | -| 8d || 556.9 | 652.0 | +17% || 0.51 | 0.49 | -5% | 0.0232 | -| 9a || 418.6 | 516.6 | +23% || 0.51 | 0.49 | -5% | 0.0129 | -| 9b || 295.8 | 360.8 | +22% || 0.51 | 0.49 | -5% | 0.0693 | -| 9c || 573.3 | 692.3 | +21% || 0.51 | 0.49 | -5% | 0.0140 | -| 9d || 893.7 | 938.2 | +5% || 0.51 | 0.49 | -4% | 0.4058 | +---------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 50603.1 | 53273.9 | +5% || | | | | -| Geomean || | | || | | -5% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

[mweisgut]

[3] RobinMap vs. Memory Tracking Disabled (Iter. 2)

Command:

../scripts/benchmark_all.sh bbe2301e731486a993dcb9b2c15b62fbed65c5c5 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8

System

nemea - click to expand

| property | value | | -- | -- | | Hostname | nemea | | CPU | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | Memory | 1.2TB | | numactl | nodebind: 1 | | numactl | membind: 1 |

Commit Info and Build Time	commit	date	message	build time
bbe2301e7	21.07.2022 13:31	Exchanged emplace with try_emplace	real 332.85 user 2770.95 sys 78.74
5acb2f2a0	22.07.2022 10:35	disabled tracking	real 329.67 user 2811.49 sys 80.18

hyriseBenchmarkTPCH - single-threaded, SF 10.0

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +2%

Configuration Overview - click to expand

```diff +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 17:20:52 | 2022-07-25 20:45:22 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 5880.2 | 5751.2 | -2% || 0.17 | 0.17 | +2% | 0.2459 | +| TPC-H 02 || 48.2 | 42.8 | -11%˄ || 20.73 | 23.37 | +13%˄ | 0.0000 | | TPC-H 03 || 2801.2 | 2755.4 | -2% || 0.36 | 0.36 | +2% | 0.0689 | | TPC-H 04 || 1627.6 | 1637.1 | +1% || 0.61 | 0.61 | -1% | 0.6110 | | TPC-H 05 || 3482.6 | 3475.1 | -0% || 0.29 | 0.29 | +0% | 0.7685 | | TPC-H 06 || 152.6 | 154.2 | +1%˄ || 6.55 | 6.49 | -1%˄ | 0.1232 | | TPC-H 07 || 1074.4 | 1031.7 | -4% || 0.93 | 0.97 | +4% | 0.0000 | +| TPC-H 08 || 827.7 | 783.3 | -5% || 1.21 | 1.28 | +6% | 0.0000 | | TPC-H 09 || 5350.4 | 5331.8 | -0% || 0.19 | 0.19 | +0% | 0.3001 | | TPC-H 10 || 3251.2 | 3154.0 | -3% || 0.31 | 0.32 | +3% | 0.0132 | +| TPC-H 11 || 78.8 | 73.2 | -7%˄ || 12.68 | 13.65 | +8%˄ | 0.0000 | | TPC-H 12 || 1118.9 | 1071.0 | -4% || 0.89 | 0.93 | +4% | 0.0000 | | TPC-H 13 || 4981.9 | 4929.1 | -1% || 0.20 | 0.20 | +1% | 0.3293 | | TPC-H 14 || 429.8 | 429.9 | +0%˄ || 2.33 | 2.33 | -0%˄ | 0.9435 | | TPC-H 15 || 192.8 | 190.4 | -1%˄ || 5.19 | 5.25 | +1%˄ | 0.0000 | | TPC-H 16 || 624.4 | 633.0 | +1% || 1.60 | 1.58 | -1% | 0.0000 | | TPC-H 17 || 220.4 | 214.3 | -3%˄ || 4.54 | 4.67 | +3%˄ | 0.0000 | | TPC-H 18 || 1538.6 | 1579.4 | +3% || 0.65 | 0.63 | -3% | 0.0000 | | TPC-H 19 || 269.4 | 266.8 | -1%˄ || 3.71 | 3.75 | +1%˄ | 0.0199 | | TPC-H 20 || 407.9 | 396.8 | -3%˄ || 2.45 | 2.52 | +3%˄ | 0.0000 | | TPC-H 21 || 5438.9 | 5363.5 | -1% || 0.18 | 0.19 | +1% | 0.0688 | | TPC-H 22 || 477.0 | 477.7 | +0%˄ || 2.10 | 2.09 | -0%˄ | 0.2739 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 40275.0 | 39741.7 | -1% || | | | | | Geomean || | | || | | +2% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - single-threaded, SF 0.01

Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +3%

Configuration Overview - click to expand

```diff +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st_s01.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st_s01.json | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 17:40:36 | 2022-07-25 21:04:58 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 0.009999999776482582 | 0.009999999776482582 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 4.9 | 4.8 | -2%˄ || 205.53 | 210.09 | +2%˄ | 0.0000 | -| TPC-H 02 || 3.6 | 3.8 | +5%˄ || 277.34 | 263.91 | -5%˄ | 0.4255 | +| TPC-H 03 || 0.7 | 0.7 | -9%˄ || 1389.96 | 1524.33 | +10%˄ | 0.0000 | +| TPC-H 04 || 0.5 | 0.4 | -11%˄ || 2051.03 | 2315.03 | +13%˄ | 0.0000 | | TPC-H 05 || 1.0 | 1.0 | -3%˄ || 962.52 | 995.15 | +3%˄ | 0.0000 | | TPC-H 06 || 0.2 | 0.2 | -0%˄ || 5670.42 | 5690.64 | +0%˄ | 0.5597 | +| TPC-H 07 || 10.8 | 9.7 | -10%˄ || 92.52 | 102.61 | +11%˄ | 0.1615 | | TPC-H 08 || 15.8 | 15.4 | -3%˄ || 63.37 | 65.10 | +3%˄ | 0.1690 | | TPC-H 09 || 3.3 | 3.3 | +1%˄ || 302.56 | 299.09 | -1%˄ | 0.9383 | +| TPC-H 10 || 0.8 | 0.8 | -5%˄ || 1223.54 | 1281.78 | +5%˄ | 0.0000 | | TPC-H 11 || 0.2 | 0.2 | -3%˄ || 4697.71 | 4831.30 | +3%˄ | 0.0178 | | TPC-H 12 || 0.6 | 0.6 | -1%˄ || 1623.21 | 1642.84 | +1%˄ | 0.0005 | | TPC-H 13 || 2.0 | 1.9 | -3%˄ || 508.22 | 521.70 | +3%˄ | 0.0000 | | TPC-H 14 || 0.3 | 0.3 | +0%˄ || 2958.44 | 2958.25 | -0%˄ | 0.9930 | | TPC-H 15 || 1.1 | 1.1 | +0%˄ || 884.59 | 883.95 | -0%˄ | 0.6104 | | TPC-H 16 || 1.9 | 1.9 | -1%˄ || 516.96 | 519.61 | +1%˄ | 0.0931 | | TPC-H 17 || 0.7 | 0.7 | +1%˄ || 1435.09 | 1421.72 | -1%˄ | 0.9328 | | TPC-H 18 || 1.2 | 1.1 | -3%˄ || 857.00 | 885.14 | +3%˄ | 0.0000 | | TPC-H 19 || 4.8 | 4.8 | -1%˄ || 207.45 | 209.57 | +1%˄ | 0.0000 | +| TPC-H 20 || 2.1 | 2.0 | -7%˄ || 467.14 | 500.96 | +7%˄ | 0.0020 | +| TPC-H 21 || 1.5 | 1.4 | -6%˄ || 674.25 | 719.60 | +7%˄ | 0.1848 | | TPC-H 22 || 1.0 | 1.0 | +1%˄ || 1018.47 | 1009.67 | -1%˄ | 0.0001 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 59.1 | 57.1 | -3% || | | | | | Geomean || | | || | | +3% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt_ordered.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt_ordered.json | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 17:41:04 | 2022-07-25 21:05:26 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | TPC-H 01 || 4820.1 | 4768.9 | -1% || 0.20 | 0.20 | +0% | (run time too short) | | TPC-H 02 || 64.6 | 62.6 | -3% || 14.21 | 14.63 | +3% | 0.0027 | | TPC-H 03 || 1193.2 | 1204.1 | +1% || 0.83 | 0.82 | -2% | 0.3428 | | TPC-H 04 || 676.2 | 664.7 | -2% || 1.47 | 1.48 | +1% | 0.1478 | | TPC-H 05 || 1039.8 | 1066.7 | +3% || 0.95 | 0.92 | -4% | (run time too short) | | TPC-H 06 || 71.9 | 71.6 | -1% || 12.95 | 13.03 | +1% | 0.1877 | | TPC-H 07 || 480.2 | 474.8 | -1% || 2.05 | 2.08 | +2% | 0.0400 | | TPC-H 08 || 442.4 | 431.2 | -3% || 2.23 | 2.28 | +2% | 0.0000 | | TPC-H 09 || 2536.9 | 2522.4 | -1% || 0.38 | 0.38 | +0% | (run time too short) | | TPC-H 10 || 1673.7 | 1676.1 | +0% || 0.58 | 0.58 | -0% | (run time too short) | | TPC-H 11 || 92.4 | 91.0 | -2% || 10.25 | 10.42 | +2% | 0.0104 | | TPC-H 12 || 593.9 | 586.2 | -1% || 1.67 | 1.68 | +1% | 0.0014 | | TPC-H 13 || 3092.4 | 3068.6 | -1% || 0.32 | 0.32 | -0% | (run time too short) | | TPC-H 14 || 173.9 | 175.3 | +1% || 5.58 | 5.53 | -1% | 0.1440 | | TPC-H 15 || 168.3 | 168.1 | -0% || 5.77 | 5.78 | +0% | 0.8411 | | TPC-H 16 || 766.2 | 775.6 | +1% || 1.28 | 1.27 | -1% | (run time too short) | | TPC-H 17 || 95.2 | 93.8 | -2% || 9.97 | 10.12 | +2% | 0.0020 | | TPC-H 18 || 2554.4 | 2597.5 | +2% || 0.38 | 0.38 | -0% | (run time too short) | | TPC-H 19 || 157.1 | 155.5 | -1% || 6.15 | 6.22 | +1% | 0.1034 | | TPC-H 20 || 251.3 | 248.9 | -1% || 3.90 | 3.93 | +1% | 0.1808 | | TPC-H 21 || 1348.8 | 1332.7 | -1% || 0.73 | 0.73 | -0% | (run time too short) | | TPC-H 22 || 174.1 | 173.5 | -0% || 5.58 | 5.58 | -0% | 0.3472 | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 22467.1 | 22409.7 | -0% || | | | | | Geomean || | | || | | +0% | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 18:04:55 | 2022-07-25 21:29:14 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 5736.3 | 5690.4 | -1% || 0.42 | 0.42 | +1% | 0.6059 | +| TPC-H 02 || 387.4 | 365.9 | -6% || 0.42 | 0.43 | +0% | 0.7658 | +| TPC-H 03 || 4211.7 | 3996.2 | -5% || 0.42 | 0.43 | +1% | 0.3273 | | TPC-H 04 || 2745.7 | 2713.7 | -1% || 0.42 | 0.43 | +1% | 0.8755 | | TPC-H 05 || 4752.9 | 4830.3 | +2% || 0.42 | 0.43 | +1% | 0.7653 | -| TPC-H 06 || 462.7 | 487.3 | +5% || 0.42 | 0.43 | +1% | 0.6543 | | TPC-H 07 || 2785.8 | 2722.6 | -2% || 0.42 | 0.43 | +1% | 0.7232 | | TPC-H 08 || 2087.5 | 2025.9 | -3% || 0.42 | 0.43 | +1% | 0.6807 | -| TPC-H 09 || 6102.2 | 6393.2 | +5% || 0.42 | 0.42 | +1% | 0.2287 | | TPC-H 10 || 5036.6 | 5067.0 | +1% || 0.42 | 0.43 | +1% | 0.8811 | -| TPC-H 11 || 562.2 | 665.8 | +18% || 0.42 | 0.43 | +1% | 0.3335 | -| TPC-H 12 || 2315.2 | 2442.8 | +6% || 0.42 | 0.43 | +1% | 0.4507 | | TPC-H 13 || 6174.9 | 5962.4 | -3% || 0.42 | 0.42 | +1% | 0.1029 | -| TPC-H 14 || 1136.4 | 1194.6 | +5% || 0.42 | 0.43 | +0% | 0.6522 | +| TPC-H 15 || 497.4 | 470.1 | -5% || 0.42 | 0.43 | +1% | 0.5308 | | TPC-H 16 || 2324.3 | 2349.1 | +1% || 0.42 | 0.43 | +1% | 0.8884 | -| TPC-H 17 || 568.8 | 608.2 | +7% || 0.42 | 0.43 | +1% | 0.6120 | | TPC-H 18 || 3649.4 | 3520.0 | -4% || 0.42 | 0.43 | +1% | 0.1467 | -| TPC-H 19 || 794.0 | 882.6 | +11% || 0.42 | 0.43 | +1% | 0.4100 | -| TPC-H 20 || 1159.2 | 1337.1 | +15% || 0.42 | 0.43 | +1% | 0.0535 | | TPC-H 21 || 6200.2 | 5930.9 | -4% || 0.42 | 0.42 | +1% | 0.2994 | | TPC-H 22 || 1043.5 | 1012.9 | -3% || 0.42 | 0.43 | +1% | 0.8288 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 60734.4 | 60669.0 | -0% || | | | | | Geomean || | | || | | +1% | | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - single-threaded

Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +10%

Configuration Overview - click to expand

```diff +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 18:26:41 | 2022-07-25 21:50:59 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 262.6 | 261.8 | -0%˄ || 3.81 | 3.82 | +0%˄ | 0.4629 | +| 03 || 79.6 | 69.4 | -13%˄ || 12.57 | 14.42 | +15%˄ | 0.0000 | +| 06 || 167.8 | 150.2 | -10%˄ || 5.96 | 6.66 | +12%˄ | 0.0000 | +| 07 || 316.1 | 295.7 | -6%˄ || 3.16 | 3.38 | +7%˄ | 0.0000 | | 09 || 630.1 | 636.3 | +1% || 1.59 | 1.57 | -1% | 0.0000 | +| 10 || 161.2 | 133.6 | -17%˄ || 6.20 | 7.49 | +21%˄ | 0.0000 | | 13 || 439.5 | 421.6 | -4%˄ || 2.28 | 2.37 | +4%˄ | 0.0000 | +| 15 || 113.5 | 108.1 | -5%˄ || 8.81 | 9.25 | +5%˄ | 0.0000 | +| 16 || 158.5 | 143.7 | -9%˄ || 6.31 | 6.96 | +10%˄ | 0.0000 | +| 17 || 337.4 | 309.6 | -8%˄ || 2.96 | 3.23 | +9%˄ | 0.0000 | +| 19 || 119.7 | 99.7 | -17%˄ || 8.36 | 10.03 | +20%˄ | 0.0000 | +| 25 || 192.5 | 167.4 | -13%˄ || 5.19 | 5.97 | +15%˄ | 0.0000 | | 26 || 134.0 | 133.5 | -0%˄ || 7.46 | 7.49 | +0%˄ | 0.4379 | | 28 || 602.9 | 595.5 | -1%˄ || 1.66 | 1.68 | +1%˄ | 0.0000 | +| 29 || 497.9 | 474.9 | -5%˄ || 2.01 | 2.11 | +5%˄ | 0.0000 | +| 31 || 1317.3 | 1249.6 | -5% || 0.76 | 0.80 | +5% | 0.0000 | +| 32 || 50.4 | 44.0 | -13%˄ || 19.85 | 22.74 | +15%˄ | 0.0000 | +| 34 || 185.0 | 163.8 | -11%˄ || 5.41 | 6.11 | +13%˄ | 0.0000 | +| 35 || 673.6 | 642.2 | -5% || 1.48 | 1.56 | +5% | 0.0000 | +| 37 || 326.0 | 267.2 | -18%˄ || 3.07 | 3.74 | +22%˄ | 0.0000 | +| 39a || 1774.5 | 1678.8 | -5% || 0.56 | 0.60 | +6% | 0.0000 | | 39b || 1757.9 | 1711.1 | -3% || 0.57 | 0.58 | +3% | 0.0001 | | 41 || 287.0 | 287.0 | -0%˄ || 3.48 | 3.48 | +0%˄ | 0.8714 | +| 42 || 100.4 | 77.4 | -23%˄ || 9.96 | 12.93 | +30%˄ | 0.0000 | | 43 || 951.2 | 934.0 | -2% || 1.05 | 1.07 | +2% | 0.0000 | +| 45 || 121.1 | 115.2 | -5%˄ || 8.26 | 8.68 | +5%˄ | 0.0000 | | 48 || 1033.3 | 1025.0 | -1% || 0.97 | 0.98 | +1% | 0.1006 | +| 50 || 129.1 | 108.7 | -16%˄ || 7.75 | 9.20 | +19%˄ | 0.0000 | +| 52 || 96.5 | 77.4 | -20%˄ || 10.36 | 12.91 | +25%˄ | 0.0000 | +| 55 || 93.7 | 74.3 | -21%˄ || 10.67 | 13.45 | +26%˄ | 0.0000 | | 62 || 530.1 | 525.1 | -1%˄ || 1.89 | 1.90 | +1%˄ | 0.0000 | | 65 || 1748.0 | 1733.5 | -1% || 0.57 | 0.58 | +1% | 0.1090 | +| 69 || 151.2 | 120.6 | -20%˄ || 6.62 | 8.29 | +25%˄ | 0.0000 | +| 73 || 94.8 | 76.1 | -20%˄ || 10.55 | 13.14 | +25%˄ | 0.0000 | | 79 || 483.7 | 465.0 | -4%˄ || 2.07 | 2.15 | +4%˄ | 0.0000 | | 81 || 206.7 | 201.9 | -2%˄ || 4.84 | 4.95 | +2%˄ | 0.0000 | +| 82 || 425.0 | 354.5 | -17%˄ || 2.35 | 2.82 | +20%˄ | 0.0000 | +| 83 || 42.9 | 39.6 | -8%˄ || 23.31 | 25.22 | +8%˄ | 0.0000 | +| 85 || 132.7 | 124.6 | -6%˄ || 7.54 | 8.03 | +7%˄ | 0.0000 | +| 88 || 799.9 | 657.6 | -18% || 1.25 | 1.52 | +22% | 0.0000 | +| 91 || 17.6 | 16.1 | -9%˄ || 56.72 | 62.00 | +9%˄ | 0.0000 | +| 92 || 40.0 | 35.6 | -11%˄ || 24.98 | 28.09 | +12%˄ | 0.0000 | | 93 || 3826.0 | 3821.8 | -0% || 0.26 | 0.26 | +0% | 0.7758 | +| 94 || 104.4 | 94.4 | -10%˄ || 9.58 | 10.60 | +11%˄ | 0.0000 | | 95 || 8998.6 | 9017.9 | +0% || 0.11 | 0.11 | -0% | ˅ | +| 96 || 76.8 | 57.2 | -26%˄ || 13.02 | 17.49 | +34%˄ | 0.0000 | | 97 || 3242.8 | 3287.6 | +1% || 0.31 | 0.30 | -1% | 0.1182 | | 99 || 1023.1 | 1018.1 | -0% || 0.98 | 0.98 | +0% | 0.0011 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 35054.5 | 34103.8 | -3% || | | | | +| Geomean || | | || | | +10% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | | || ˅ Insufficient number of runs for p-value calculation | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +2%

Configuration Overview - click to expand

```diff +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 18:53:37 | 2022-07-25 22:16:55 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 651.2 | 635.3 | -2% || 0.46 | 0.47 | +2% | 0.8209 | -| 03 || 198.4 | 238.8 | +20% || 0.46 | 0.47 | +2% | 0.4031 | +| 06 || 738.4 | 587.1 | -20% || 0.46 | 0.47 | +2% | 0.1147 | | 07 || 835.9 | 824.7 | -1% || 0.46 | 0.47 | +2% | 0.9047 | -| 09 || 1218.8 | 1285.1 | +5% || 0.46 | 0.47 | +2% | 0.3023 | +| 10 || 571.2 | 477.0 | -16% || 0.46 | 0.47 | +1% | 0.3302 | | 13 || 1577.9 | 1518.2 | -4% || 0.46 | 0.47 | +2% | 0.6849 | -| 15 || 451.6 | 504.6 | +12% || 0.46 | 0.47 | +2% | 0.4983 | | 16 || 632.3 | 648.2 | +3% || 0.46 | 0.47 | +2% | 0.8673 | -| 17 || 1001.3 | 1104.7 | +10% || 0.46 | 0.47 | +2% | 0.4070 | | 19 || 420.9 | 439.7 | +4% || 0.46 | 0.47 | +2% | 0.7507 | +| 25 || 794.8 | 666.0 | -16% || 0.46 | 0.47 | +2% | 0.1647 | -| 26 || 484.1 | 512.5 | +6% || 0.46 | 0.47 | +2% | 0.6925 | +| 28 || 612.5 | 570.3 | -7% || 0.46 | 0.47 | +2% | 0.6214 | +| 29 || 1420.5 | 1340.3 | -6% || 0.46 | 0.47 | +2% | 0.4964 | -| 31 || 1803.0 | 1886.1 | +5% || 0.46 | 0.47 | +2% | 0.5576 | +| 32 || 355.7 | 212.1 | -40% || 0.46 | 0.47 | +2% | 0.0232 | +| 34 || 508.0 | 434.3 | -15% || 0.46 | 0.47 | +2% | 0.2331 | -| 35 || 1953.3 | 2053.5 | +5% || 0.46 | 0.47 | +2% | 0.5047 | | 37 || 575.8 | 551.3 | -4% || 0.46 | 0.47 | +1% | 0.7180 | | 39a || 2089.5 | 2175.4 | +4% || 0.46 | 0.47 | +2% | 0.4649 | | 39b || 2098.3 | 2162.0 | +3% || 0.46 | 0.47 | +2% | 0.6101 | | 41 || 2827.2 | 2860.0 | +1% || 0.46 | 0.47 | +2% | 0.8461 | +| 42 || 368.9 | 281.7 | -24% || 0.46 | 0.47 | +2% | 0.1363 | +| 43 || 1454.7 | 1370.6 | -6% || 0.46 | 0.47 | +2% | 0.4175 | | 45 || 539.0 | 530.1 | -2% || 0.46 | 0.47 | +2% | 0.9231 | +| 48 || 2715.4 | 2548.8 | -6% || 0.46 | 0.47 | +2% | 0.3147 | +| 50 || 583.9 | 482.0 | -17% || 0.46 | 0.47 | +2% | 0.1349 | | 52 || 331.4 | 342.9 | +3% || 0.46 | 0.47 | +2% | 0.8312 | +| 55 || 305.8 | 270.0 | -12% || 0.46 | 0.47 | +2% | 0.4780 | -| 62 || 847.9 | 893.0 | +5% || 0.46 | 0.47 | +2% | 0.5582 | | 65 || 3701.4 | 3701.7 | +0% || 0.46 | 0.47 | +2% | 0.9974 | +| 69 || 716.8 | 598.7 | -16% || 0.46 | 0.47 | +2% | 0.2734 | | 73 || 269.4 | 271.1 | +1% || 0.46 | 0.47 | +2% | 0.9747 | | 79 || 1205.0 | 1166.3 | -3% || 0.46 | 0.47 | +2% | 0.6842 | | 81 || 797.3 | 769.6 | -3% || 0.46 | 0.47 | +2% | 0.7440 | -| 82 || 773.3 | 931.1 | +20% || 0.46 | 0.47 | +2% | 0.1508 | | 83 || 240.7 | 232.3 | -3% || 0.46 | 0.47 | +2% | 0.8520 | | 85 || 634.9 | 659.7 | +4% || 0.46 | 0.47 | +2% | 0.7433 | | 88 || 744.0 | 776.5 | +4% || 0.46 | 0.47 | +2% | 0.6902 | -| 91 || 146.8 | 222.6 | +52% || 0.46 | 0.47 | +2% | 0.1647 | +| 92 || 261.6 | 234.5 | -10% || 0.46 | 0.47 | +2% | 0.5531 | | 93 || 2324.3 | 2380.0 | +2% || 0.46 | 0.47 | +2% | 0.6785 | +| 94 || 733.0 | 670.2 | -9% || 0.46 | 0.47 | +2% | 0.5585 | | 95 || 8165.8 | 7875.2 | -4% || 0.46 | 0.47 | +2% | 0.1046 | +| 96 || 197.6 | 176.8 | -11% || 0.46 | 0.47 | +2% | 0.5178 | | 97 || 4311.2 | 4412.1 | +2% || 0.46 | 0.47 | +2% | 0.5356 | +| 99 || 1362.8 | 1261.6 | -7% || 0.46 | 0.47 | +2% | 0.2261 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 57553.4 | 56746.4 | -1% || | | | | | Geomean || | | || | | +2% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - single-threaded

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 19:14:04 | 2022-07-25 22:37:23 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 22.9 | 23.1 | +1% || 4.27 | 4.25 | -0% | 0.0358 | | New-Order || 16.4 | 16.4 | +0% || 47.94 | 47.92 | -0% | 0.8787 | | Order-Status || 1.1 | 1.1 | -0% || 4.27 | 4.27 | -0% | 0.9646 | | Payment || 2.1 | 2.0 | -1% || 45.81 | 45.72 | -0% | 0.0354 | | Stock-Level || 3.6 | 3.6 | -1% || 4.27 | 4.27 | -0% | 0.1531 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 46.0 | 46.2 | +0% || | | | | | Geomean || | | || | | -0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 19:15:05 | 2022-07-25 22:38:24 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 219.2 | 215.7 | -2% || 4.32 | 4.39 | +2% | 0.0292 | | unsucc.: || 4.3 | 4.3 | +0% || 62.89 | 62.89 | +0% | | | New-Order || 57.2 | 57.0 | -0% || 134.42 | 134.55 | +0% | 0.0654 | | unsucc.: || 4.9 | 4.9 | +1% || 621.63 | 622.35 | +0% | | | Order-Status || 7.5 | 7.6 | +1% || 67.20 | 67.28 | +0% | 0.0106 | | Payment || 9.9 | 9.9 | +0% || 21.40 | 21.47 | +0% | 0.7617 | | unsucc.: || 4.4 | 4.4 | +1% || 701.05 | 701.80 | +0% | | | Stock-Level || 17.6 | 17.3 | -2% || 67.20 | 67.28 | +0% | 0.0000 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 311.4 | 307.5 | -1% || | | | | | Geomean || | | || | | +0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - single-threaded

Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +14%

Configuration Overview - click to expand

```diff +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 19:35:12 | 2022-07-25 22:58:31 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ +| 10a || 177.8 | 164.2 | -8%˄ || 5.62 | 6.09 | +8%˄ | 0.0000 | +| 10b || 128.0 | 114.7 | -10%˄ || 7.81 | 8.72 | +12%˄ | 0.0000 | | 10c || 322.5 | 315.3 | -2%˄ || 3.10 | 3.17 | +2%˄ | 0.0001 | +| 11a || 20.1 | 15.1 | -25%˄ || 49.83 | 66.37 | +33%˄ | 0.0000 | +| 11b || 20.0 | 14.8 | -26%˄ || 50.00 | 67.33 | +35%˄ | 0.0000 | +| 11c || 25.3 | 20.2 | -20%˄ || 39.57 | 49.45 | +25%˄ | 0.0000 | +| 11d || 28.0 | 23.0 | -18%˄ || 35.73 | 43.56 | +22%˄ | 0.0000 | +| 12a || 50.2 | 46.4 | -8%˄ || 19.91 | 21.54 | +8%˄ | 0.0000 | +| 12b || 40.5 | 29.1 | -28%˄ || 24.70 | 34.38 | +39%˄ | 0.0000 | +| 12c || 87.5 | 83.2 | -5%˄ || 11.43 | 12.02 | +5%˄ | 0.0000 | +| 13a || 286.8 | 268.5 | -6%˄ || 3.49 | 3.72 | +7%˄ | 0.0000 | +| 13b || 170.0 | 155.0 | -9%˄ || 5.88 | 6.45 | +10%˄ | 0.0000 | +| 13c || 132.6 | 118.3 | -11%˄ || 7.54 | 8.45 | +12%˄ | 0.0000 | | 13d || 576.3 | 556.7 | -3%˄ || 1.74 | 1.80 | +4%˄ | 0.0000 | | 14a || 424.4 | 412.8 | -3%˄ || 2.36 | 2.42 | +3%˄ | 0.0000 | | 14b || 252.3 | 245.7 | -3%˄ || 3.96 | 4.07 | +3%˄ | 0.0000 | | 14c || 500.3 | 491.2 | -2%˄ || 2.00 | 2.04 | +2%˄ | 0.0000 | +| 15a || 89.3 | 79.1 | -11%˄ || 11.20 | 12.65 | +13%˄ | 0.0000 | +| 15b || 88.5 | 78.5 | -11%˄ || 11.30 | 12.75 | +13%˄ | 0.0000 | +| 15c || 93.6 | 83.6 | -11%˄ || 10.69 | 11.96 | +12%˄ | 0.0000 | +| 15d || 90.8 | 79.8 | -12%˄ || 11.01 | 12.53 | +14%˄ | 0.0000 | | 16a || 2308.1 | 2292.2 | -1% || 0.43 | 0.44 | +1% | 0.1433 | | 16b || 3353.1 | 3327.8 | -1% || 0.30 | 0.30 | +1% | 0.0742 | | 16c || 2404.0 | 2372.2 | -1% || 0.42 | 0.42 | +1% | 0.0003 | | 16d || 2380.2 | 2338.1 | -2% || 0.42 | 0.43 | +2% | 0.0000 | | 17a || 616.5 | 596.8 | -3%˄ || 1.62 | 1.68 | +3%˄ | 0.0000 | +| 17b || 495.9 | 474.1 | -4%˄ || 2.02 | 2.11 | +5%˄ | 0.0000 | +| 17c || 473.6 | 451.9 | -5%˄ || 2.11 | 2.21 | +5%˄ | 0.0000 | | 17d || 554.1 | 530.8 | -4%˄ || 1.80 | 1.88 | +4%˄ | 0.0000 | | 17e || 1616.6 | 1587.1 | -2% || 0.62 | 0.63 | +2% | 0.0000 | | 17f || 980.9 | 954.9 | -3% || 1.02 | 1.05 | +3% | 0.0000 | | 18a || 641.3 | 627.1 | -2% || 1.56 | 1.59 | +2% | 0.0000 | +| 18b || 123.0 | 110.2 | -10%˄ || 8.13 | 9.07 | +12%˄ | 0.0000 | | 18c || 580.8 | 556.2 | -4%˄ || 1.72 | 1.80 | +4%˄ | 0.0000 | +| 19a || 193.3 | 178.7 | -8%˄ || 5.17 | 5.60 | +8%˄ | 0.0000 | +| 19b || 136.7 | 122.1 | -11%˄ || 7.32 | 8.19 | +12%˄ | 0.0000 | +| 19c || 205.6 | 190.8 | -7%˄ || 4.86 | 5.24 | +8%˄ | 0.0000 | | 19d || 736.5 | 722.8 | -2% || 1.36 | 1.38 | +2% | 0.0000 | +| 1a || 12.0 | 8.8 | -26%˄ || 83.67 | 113.46 | +36%˄ | 0.0000 | +| 1b || 11.4 | 8.2 | -28%˄ || 87.72 | 121.37 | +38%˄ | 0.0000 | +| 1c || 12.5 | 9.3 | -26%˄ || 80.14 | 107.98 | +35%˄ | 0.0000 | +| 1d || 11.4 | 8.3 | -28%˄ || 87.44 | 120.88 | +38%˄ | 0.0000 | | 20a || 401.4 | 384.7 | -4%˄ || 2.49 | 2.60 | +4%˄ | 0.0000 | +| 20b || 242.6 | 225.2 | -7%˄ || 4.12 | 4.44 | +8%˄ | 0.0000 | +| 20c || 234.2 | 218.1 | -7%˄ || 4.27 | 4.59 | +7%˄ | 0.0000 | +| 21a || 48.4 | 35.0 | -28%˄ || 20.66 | 28.60 | +38%˄ | 0.0000 | +| 21b || 24.2 | 18.8 | -22%˄ || 41.33 | 53.23 | +29%˄ | 0.0000 | +| 21c || 48.0 | 34.4 | -28%˄ || 20.84 | 29.05 | +39%˄ | 0.0000 | +| 22a || 215.1 | 205.1 | -5%˄ || 4.65 | 4.88 | +5%˄ | 0.0000 | +| 22b || 156.3 | 146.2 | -6%˄ || 6.40 | 6.84 | +7%˄ | 0.0000 | | 22c || 514.6 | 499.4 | -3%˄ || 1.94 | 2.00 | +3%˄ | 0.0000 | | 22d || 927.6 | 921.1 | -1% || 1.08 | 1.09 | +1% | 0.0368 | +| 23a || 54.7 | 42.0 | -23%˄ || 18.29 | 23.81 | +30%˄ | 0.0000 | +| 23b || 55.3 | 43.5 | -21%˄ || 18.07 | 23.00 | +27%˄ | 0.0000 | +| 23c || 93.7 | 83.1 | -11%˄ || 10.68 | 12.03 | +13%˄ | 0.0000 | +| 24a || 167.9 | 142.8 | -15%˄ || 5.96 | 7.00 | +18%˄ | 0.0000 | +| 24b || 147.2 | 121.9 | -17%˄ || 6.79 | 8.20 | +21%˄ | 0.0000 | +| 25a || 291.6 | 273.9 | -6%˄ || 3.43 | 3.65 | +6%˄ | 0.0000 | +| 25b || 135.9 | 112.3 | -17%˄ || 7.36 | 8.90 | +21%˄ | 0.0000 | | 25c || 932.4 | 913.1 | -2% || 1.07 | 1.10 | +2% | 0.0000 | +| 26a || 178.3 | 162.0 | -9%˄ || 5.61 | 6.17 | +10%˄ | 0.0000 | +| 26b || 146.9 | 129.7 | -12%˄ || 6.81 | 7.71 | +13%˄ | 0.0000 | +| 26c || 247.4 | 230.4 | -7%˄ || 4.04 | 4.34 | +7%˄ | 0.0000 | +| 27a || 27.8 | 22.6 | -19%˄ || 36.00 | 44.25 | +23%˄ | 0.0000 | +| 27b || 27.4 | 21.8 | -20%˄ || 36.45 | 45.79 | +26%˄ | 0.0000 | +| 27c || 48.2 | 34.7 | -28%˄ || 20.76 | 28.85 | +39%˄ | 0.0000 | | 28a || 304.6 | 295.1 | -3%˄ || 3.28 | 3.39 | +3%˄ | 0.0000 | +| 28b || 55.3 | 48.9 | -12%˄ || 18.09 | 20.46 | +13%˄ | 0.0000 | | 28c || 347.4 | 335.3 | -3%˄ || 2.88 | 2.98 | +4%˄ | 0.0000 | +| 29a || 133.3 | 101.5 | -24%˄ || 7.50 | 9.85 | +31%˄ | 0.0000 | +| 29b || 106.4 | 75.9 | -29%˄ || 9.40 | 13.17 | +40%˄ | 0.0000 | +| 29c || 174.8 | 147.4 | -16%˄ || 5.72 | 6.79 | +19%˄ | 0.0000 | +| 2a || 50.9 | 45.6 | -11%˄ || 19.63 | 21.93 | +12%˄ | 0.0000 | +| 2b || 42.6 | 37.4 | -12%˄ || 23.48 | 26.73 | +14%˄ | 0.0000 | +| 2c || 26.9 | 22.0 | -18%˄ || 37.19 | 45.48 | +22%˄ | 0.0000 | +| 2d || 69.8 | 64.7 | -7%˄ || 14.32 | 15.46 | +8%˄ | 0.0000 | +| 30a || 163.9 | 147.5 | -10%˄ || 6.10 | 6.78 | +11%˄ | 0.0000 | +| 30b || 142.9 | 126.0 | -12%˄ || 7.00 | 7.94 | +13%˄ | 0.0000 | +| 30c || 398.9 | 380.4 | -5%˄ || 2.51 | 2.63 | +5%˄ | 0.0000 | +| 31a || 171.6 | 144.8 | -16%˄ || 5.83 | 6.91 | +19%˄ | 0.0000 | +| 31b || 150.1 | 123.9 | -17%˄ || 6.66 | 8.07 | +21%˄ | 0.0000 | +| 31c || 195.5 | 168.9 | -14%˄ || 5.11 | 5.92 | +16%˄ | 0.0000 | +| 32a || 16.9 | 12.8 | -24%˄ || 59.21 | 78.15 | +32%˄ | 0.0000 | +| 32b || 37.0 | 32.6 | -12%˄ || 27.04 | 30.68 | +13%˄ | 0.0000 | +| 33a || 26.1 | 19.0 | -27%˄ || 38.35 | 52.56 | +37%˄ | 0.0000 | +| 33b || 25.5 | 18.4 | -28%˄ || 39.18 | 54.30 | +39%˄ | 0.0000 | +| 33c || 29.3 | 22.3 | -24%˄ || 34.11 | 44.89 | +32%˄ | 0.0000 | +| 3a || 126.9 | 120.8 | -5%˄ || 7.88 | 8.27 | +5%˄ | 0.0000 | +| 3b || 16.3 | 12.4 | -24%˄ || 61.39 | 80.74 | +32%˄ | 0.0000 | | 3c || 461.6 | 445.7 | -3%˄ || 2.17 | 2.24 | +4%˄ | 0.0000 | +| 4a || 107.2 | 101.9 | -5%˄ || 9.33 | 9.82 | +5%˄ | 0.0000 | +| 4b || 15.2 | 11.3 | -26%˄ || 65.61 | 88.78 | +35%˄ | 0.0000 | | 4c || 122.9 | 118.1 | -4%˄ || 8.13 | 8.47 | +4%˄ | 0.0000 | +| 5a || 53.4 | 50.9 | -5%˄ || 18.74 | 19.65 | +5%˄ | 0.0000 | | 5b || 125.3 | 120.5 | -4%˄ || 7.98 | 8.30 | +4%˄ | 0.0000 | | 5c || 193.7 | 188.2 | -3%˄ || 5.16 | 5.31 | +3%˄ | 0.0000 | +| 6a || 117.9 | 102.5 | -13%˄ || 8.48 | 9.75 | +15%˄ | 0.0000 | +| 6b || 129.2 | 114.5 | -11%˄ || 7.74 | 8.73 | +13%˄ | 0.0000 | +| 6c || 112.9 | 97.7 | -14%˄ || 8.86 | 10.24 | +16%˄ | 0.0000 | | 6d || 550.5 | 527.6 | -4%˄ || 1.82 | 1.90 | +4%˄ | 0.0000 | +| 6e || 117.9 | 102.3 | -13%˄ || 8.48 | 9.78 | +15%˄ | 0.0000 | | 6f || 714.8 | 697.5 | -2% || 1.40 | 1.43 | +2% | 0.0000 | +| 7a || 77.8 | 52.9 | -32%˄ || 12.86 | 18.89 | +47%˄ | 0.0000 | +| 7b || 74.5 | 49.7 | -33%˄ || 13.42 | 20.13 | +50%˄ | 0.0000 | | 7c || 593.8 | 571.2 | -4%˄ || 1.68 | 1.75 | +4%˄ | 0.0000 | +| 8a || 43.0 | 37.3 | -13%˄ || 23.28 | 26.79 | +15%˄ | 0.0000 | +| 8b || 40.7 | 35.4 | -13%˄ || 24.56 | 28.28 | +15%˄ | 0.0000 | | 8c || 1928.0 | 1907.9 | -1% || 0.52 | 0.52 | +1% | 0.0000 | +| 8d || 346.8 | 324.9 | -6%˄ || 2.88 | 3.08 | +7%˄ | 0.0000 | | 9a || 191.5 | 184.8 | -4%˄ || 5.22 | 5.41 | +4%˄ | 0.0000 | +| 9b || 108.7 | 102.0 | -6%˄ || 9.20 | 9.80 | +7%˄ | 0.0000 | | 9c || 273.8 | 265.5 | -3%˄ || 3.65 | 3.77 | +3%˄ | 0.0000 | | 9d || 479.0 | 472.1 | -1%˄ || 2.09 | 2.12 | +1%˄ | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 36612.0 | 35071.4 | -4% || | | | | +| Geomean || | | || | | +14% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: -2% || Geometric mean of throughput changes: +3%

Configuration Overview - click to expand

```diff +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-25 20:19:24 | 2022-07-25 23:40:39 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ +| 10a || 192.3 | 147.1 | -24% || 0.47 | 0.49 | +3% | 0.0288 | -| 10b || 163.1 | 172.5 | +6% || 0.47 | 0.49 | +3% | 0.7448 | | 10c || 427.2 | 420.2 | -2% || 0.47 | 0.49 | +3% | 0.8137 | +| 11a || 54.7 | 44.5 | -19% || 0.47 | 0.49 | +3% | 0.3591 | -| 11b || 40.0 | 48.2 | +20% || 0.47 | 0.49 | +3% | 0.4767 | -| 11c || 60.3 | 75.2 | +25% || 0.47 | 0.49 | +3% | 0.3783 | +| 11d || 106.2 | 66.9 | -37% || 0.47 | 0.49 | +3% | 0.0451 | | 12a || 193.4 | 187.0 | -3% || 0.47 | 0.49 | +3% | 0.8110 | +| 12b || 71.3 | 57.3 | -20% || 0.47 | 0.49 | +3% | 0.3504 | -| 12c || 325.7 | 346.8 | +6% || 0.47 | 0.49 | +3% | 0.5364 | | 13a || 626.8 | 637.6 | +2% || 0.47 | 0.49 | +3% | 0.7908 | +| 13b || 319.4 | 254.9 | -20% || 0.47 | 0.49 | +3% | 0.0504 | +| 13c || 281.0 | 236.9 | -16% || 0.47 | 0.49 | +3% | 0.1447 | | 13d || 1018.1 | 1008.9 | -1% || 0.47 | 0.49 | +3% | 0.8638 | +| 14a || 837.9 | 784.3 | -6% || 0.47 | 0.49 | +3% | 0.2659 | | 14b || 510.1 | 490.3 | -4% || 0.47 | 0.49 | +3% | 0.5996 | | 14c || 1008.6 | 973.9 | -3% || 0.47 | 0.49 | +3% | 0.5512 | | 15a || 173.4 | 169.8 | -2% || 0.47 | 0.49 | +3% | 0.8870 | | 15b || 134.0 | 130.2 | -3% || 0.47 | 0.49 | +3% | 0.8296 | -| 15c || 155.9 | 167.3 | +7% || 0.47 | 0.49 | +3% | 0.6235 | +| 15d || 237.7 | 216.2 | -9% || 0.47 | 0.49 | +3% | 0.5320 | +| 16a || 2707.6 | 2559.9 | -5% || 0.47 | 0.49 | +3% | 0.0166 | | 16b || 3931.4 | 3915.6 | -0% || 0.47 | 0.49 | +3% | 0.8012 | | 16c || 2776.0 | 2847.7 | +3% || 0.47 | 0.49 | +3% | 0.2597 | | 16d || 2771.0 | 2770.8 | -0% || 0.47 | 0.49 | +3% | 0.9971 | | 17a || 850.9 | 830.6 | -2% || 0.47 | 0.49 | +3% | 0.6900 | -| 17b || 581.4 | 723.7 | +24% || 0.47 | 0.49 | +3% | 0.0042 | -| 17c || 479.4 | 577.0 | +20% || 0.47 | 0.49 | +3% | 0.0176 | +| 17d || 636.7 | 591.7 | -7% || 0.47 | 0.49 | +3% | 0.2983 | | 17e || 1598.9 | 1597.8 | -0% || 0.47 | 0.49 | +3% | 0.9850 | | 17f || 1101.6 | 1087.8 | -1% || 0.47 | 0.49 | +3% | 0.8119 | | 18a || 785.8 | 815.2 | +4% || 0.47 | 0.49 | +3% | 0.5673 | +| 18b || 141.9 | 131.5 | -7% || 0.47 | 0.49 | +3% | 0.3976 | +| 18c || 728.2 | 694.3 | -5% || 0.47 | 0.49 | +3% | 0.4904 | | 19a || 483.2 | 464.6 | -4% || 0.47 | 0.49 | +3% | 0.6969 | +| 19b || 357.1 | 333.2 | -7% || 0.47 | 0.49 | +3% | 0.5078 | +| 19c || 556.2 | 473.2 | -15% || 0.47 | 0.49 | +3% | 0.0860 | +| 19d || 1315.4 | 1223.8 | -7% || 0.47 | 0.49 | +3% | 0.0978 | | 1a || 47.4 | 46.8 | -1% || 0.47 | 0.49 | +3% | 0.9685 | -| 1b || 34.3 | 38.2 | +11% || 0.47 | 0.49 | +3% | 0.7361 | -| 1c || 33.2 | 45.1 | +36% || 0.47 | 0.49 | +3% | 0.2791 | +| 1d || 34.6 | 26.6 | -23% || 0.47 | 0.49 | +3% | 0.2376 | -| 20a || 558.8 | 586.5 | +5% || 0.47 | 0.49 | +3% | 0.4942 | | 20b || 428.8 | 417.5 | -3% || 0.47 | 0.49 | +3% | 0.7531 | +| 20c || 398.3 | 380.0 | -5% || 0.47 | 0.49 | +3% | 0.5546 | +| 21a || 86.6 | 78.9 | -9% || 0.47 | 0.49 | +3% | 0.6198 | -| 21b || 64.3 | 68.1 | +6% || 0.47 | 0.49 | +3% | 0.6900 | +| 21c || 93.1 | 66.4 | -29% || 0.47 | 0.49 | +3% | 0.0877 | +| 22a || 585.0 | 546.2 | -7% || 0.47 | 0.49 | +3% | 0.3796 | | 22b || 360.8 | 353.1 | -2% || 0.47 | 0.49 | +3% | 0.8324 | | 22c || 924.3 | 941.3 | +2% || 0.47 | 0.49 | +3% | 0.7649 | +| 22d || 1342.4 | 1269.3 | -5% || 0.47 | 0.49 | +3% | 0.2266 | +| 23a || 210.2 | 195.5 | -7% || 0.47 | 0.49 | +3% | 0.6673 | +| 23b || 131.8 | 115.0 | -13% || 0.47 | 0.49 | +3% | 0.3872 | +| 23c || 186.7 | 154.9 | -17% || 0.47 | 0.49 | +3% | 0.2337 | -| 24a || 238.0 | 256.7 | +8% || 0.47 | 0.49 | +3% | 0.5056 | +| 24b || 176.2 | 164.4 | -7% || 0.47 | 0.49 | +3% | 0.6320 | | 25a || 559.3 | 545.0 | -3% || 0.47 | 0.49 | +3% | 0.7689 | +| 25b || 153.2 | 130.4 | -15% || 0.47 | 0.49 | +3% | 0.2678 | | 25c || 999.2 | 1015.3 | +2% || 0.47 | 0.49 | +3% | 0.7744 | | 26a || 310.6 | 301.9 | -3% || 0.47 | 0.49 | +3% | 0.7712 | +| 26b || 247.2 | 214.0 | -13% || 0.47 | 0.49 | +3% | 0.2993 | | 26c || 491.2 | 490.5 | -0% || 0.47 | 0.49 | +3% | 0.9872 | +| 27a || 111.5 | 93.5 | -16% || 0.47 | 0.49 | +3% | 0.4044 | | 27b || 86.6 | 87.1 | +1% || 0.47 | 0.49 | +3% | 0.9747 | -| 27c || 71.2 | 91.2 | +28% || 0.47 | 0.49 | +3% | 0.1534 | | 28a || 719.9 | 749.5 | +4% || 0.47 | 0.49 | +3% | 0.5410 | +| 28b || 201.7 | 171.1 | -15% || 0.47 | 0.49 | +3% | 0.2425 | | 28c || 749.4 | 732.9 | -2% || 0.47 | 0.49 | +3% | 0.7335 | +| 29a || 221.6 | 192.2 | -13% || 0.47 | 0.49 | +3% | 0.2396 | +| 29b || 135.2 | 124.3 | -8% || 0.47 | 0.49 | +3% | 0.4233 | +| 29c || 263.8 | 222.5 | -16% || 0.47 | 0.49 | +3% | 0.1341 | -| 2a || 176.9 | 186.0 | +5% || 0.47 | 0.49 | +3% | 0.7723 | +| 2b || 155.5 | 142.3 | -9% || 0.47 | 0.49 | +3% | 0.5831 | -| 2c || 81.9 | 97.4 | +19% || 0.47 | 0.49 | +3% | 0.4363 | -| 2d || 169.7 | 188.9 | +11% || 0.47 | 0.49 | +3% | 0.3992 | +| 30a || 323.3 | 293.7 | -9% || 0.47 | 0.49 | +3% | 0.3237 | +| 30b || 279.5 | 230.5 | -18% || 0.47 | 0.49 | +3% | 0.1159 | -| 30c || 716.0 | 802.4 | +12% || 0.47 | 0.49 | +3% | 0.1638 | +| 31a || 358.8 | 287.5 | -20% || 0.47 | 0.49 | +3% | 0.0651 | +| 31b || 164.5 | 150.6 | -8% || 0.47 | 0.49 | +3% | 0.4965 | +| 31c || 378.2 | 306.0 | -19% || 0.47 | 0.49 | +3% | 0.0541 | | 32a || 36.9 | 37.7 | +2% || 0.47 | 0.49 | +3% | 0.9350 | -| 32b || 119.6 | 139.4 | +17% || 0.47 | 0.49 | +3% | 0.2877 | +| 33a || 95.0 | 80.3 | -15% || 0.47 | 0.49 | +3% | 0.4821 | | 33b || 71.5 | 71.1 | -1% || 0.47 | 0.49 | +3% | 0.9797 | -| 33c || 69.4 | 76.3 | +10% || 0.47 | 0.49 | +3% | 0.6034 | -| 3a || 351.9 | 431.1 | +23% || 0.47 | 0.49 | +3% | 0.1073 | +| 3b || 71.7 | 53.6 | -25% || 0.47 | 0.49 | +3% | 0.3017 | +| 3c || 608.5 | 507.8 | -17% || 0.47 | 0.49 | +3% | 0.0189 | +| 4a || 236.4 | 179.1 | -24% || 0.47 | 0.49 | +3% | 0.0505 | -| 4b || 37.2 | 52.1 | +40% || 0.47 | 0.49 | +3% | 0.1632 | | 4c || 220.2 | 229.5 | +4% || 0.47 | 0.49 | +3% | 0.7223 | -| 5a || 178.7 | 190.3 | +6% || 0.47 | 0.49 | +3% | 0.7008 | +| 5b || 218.9 | 200.5 | -8% || 0.47 | 0.49 | +3% | 0.4819 | +| 5c || 256.3 | 218.6 | -15% || 0.47 | 0.49 | +3% | 0.1830 | | 6a || 110.6 | 111.4 | +1% || 0.47 | 0.49 | +3% | 0.9643 | -| 6b || 134.6 | 146.3 | +9% || 0.47 | 0.49 | +3% | 0.4931 | +| 6c || 110.5 | 91.4 | -17% || 0.47 | 0.49 | +3% | 0.2843 | +| 6d || 534.2 | 508.6 | -5% || 0.47 | 0.49 | +3% | 0.4155 | -| 6e || 100.4 | 124.6 | +24% || 0.47 | 0.49 | +3% | 0.2094 | | 6f || 957.8 | 923.0 | -4% || 0.47 | 0.49 | +3% | 0.3074 | -| 7a || 112.8 | 131.2 | +16% || 0.47 | 0.49 | +3% | 0.3272 | +| 7b || 146.6 | 90.2 | -38% || 0.47 | 0.49 | +3% | 0.0040 | | 7c || 905.5 | 930.5 | +3% || 0.47 | 0.49 | +3% | 0.6546 | -| 8a || 137.1 | 144.2 | +5% || 0.47 | 0.49 | +3% | 0.7679 | +| 8b || 172.5 | 142.4 | -17% || 0.47 | 0.49 | +3% | 0.3713 | | 8c || 2388.6 | 2309.0 | -3% || 0.47 | 0.49 | +3% | 0.2071 | +| 8d || 672.3 | 628.7 | -6% || 0.47 | 0.49 | +3% | 0.3272 | | 9a || 491.6 | 509.8 | +4% || 0.47 | 0.49 | +3% | 0.6806 | -| 9b || 334.0 | 391.8 | +17% || 0.47 | 0.49 | +3% | 0.1817 | | 9c || 707.9 | 695.4 | -2% || 0.47 | 0.49 | +3% | 0.8269 | +| 9d || 934.2 | 871.2 | -7% || 0.47 | 0.49 | +3% | 0.1615 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 54323.7 | 53088.4 | -2% || | | | | | Geomean || | | || | | +3% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

[mweisgut]

[4] RobinMap vs. Memory Tracking Disabled (Iter. 1)

As the 2nd iteration for 1c24c5f49719e589324037c5037866a7388a034c (Hyrise Master) is currently running, the comparison between bbe2301e731486a993dcb9b2c15b62fbed65c5c5 (RobinMap) and 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8 (Memory Tracking Disabled) is shown below.

Command:

../scripts/benchmark_all.sh bbe2301e731486a993dcb9b2c15b62fbed65c5c5 

System

nemea - click to expand

| property | value | | -- | -- | | Hostname | nemea | | CPU | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | Memory | 1.2TB | | numactl | nodebind: 1 | | numactl | membind: 1 |

Commit Info and Build Time	commit	date	message	build time
bbe2301e7	21.07.2022 13:31	Exchanged emplace with try_emplace	real 329.07 user 2770.45 sys 78.28
5acb2f2a0	22.07.2022 10:35	disabled tracking	real 330.22 user 2783.27 sys 78.37

hyriseBenchmarkTPCH - single-threaded, SF 10.0

Sum of avg. item runtimes: -3% || Geometric mean of throughput changes: +3%

Configuration Overview - click to expand

```diff +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 11:44:41 | 2022-07-23 11:09:20 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ +| TPC-H 01 || 5791.7 | 5139.3 | -11% || 0.17 | 0.19 | +13% | 0.0000 | +| TPC-H 02 || 47.9 | 43.3 | -10%˄ || 20.88 | 23.11 | +11%˄ | 0.0000 | +| TPC-H 03 || 2849.7 | 2630.0 | -8% || 0.35 | 0.38 | +8% | 0.0000 | +| TPC-H 04 || 1618.7 | 1512.0 | -7% || 0.62 | 0.66 | +7% | 0.0000 | +| TPC-H 05 || 3493.3 | 3318.7 | -5% || 0.29 | 0.30 | +5% | 0.0000 | | TPC-H 06 || 153.4 | 150.4 | -2%˄ || 6.52 | 6.65 | +2%˄ | 0.0007 | +| TPC-H 07 || 1053.7 | 970.9 | -8% || 0.95 | 1.03 | +9% | 0.0000 | +| TPC-H 08 || 821.5 | 780.1 | -5% || 1.22 | 1.28 | +5% | 0.0000 | | TPC-H 09 || 5299.7 | 5375.4 | +1% || 0.19 | 0.19 | -1% | 0.0803 | | TPC-H 10 || 3167.2 | 3105.2 | -2% || 0.32 | 0.32 | +2% | 0.0315 | +| TPC-H 11 || 78.4 | 73.0 | -7%˄ || 12.75 | 13.71 | +7%˄ | 0.0000 | +| TPC-H 12 || 1112.2 | 1058.7 | -5% || 0.90 | 0.94 | +5% | 0.0000 | -| TPC-H 13 || 4943.3 | 5376.6 | +9% || 0.20 | 0.19 | -8% | 0.0000 | | TPC-H 14 || 429.4 | 433.2 | +1%˄ || 2.33 | 2.31 | -1%˄ | 0.0049 | | TPC-H 15 || 188.6 | 189.5 | +0%˄ || 5.30 | 5.28 | -0%˄ | 0.0000 | | TPC-H 16 || 625.1 | 631.4 | +1% || 1.60 | 1.58 | -1% | 0.0000 | | TPC-H 17 || 220.1 | 216.8 | -1%˄ || 4.54 | 4.61 | +2%˄ | 0.0000 | +| TPC-H 18 || 1581.7 | 1493.8 | -6% || 0.63 | 0.67 | +6% | 0.0000 | | TPC-H 19 || 270.2 | 269.8 | -0%˄ || 3.70 | 3.71 | +0%˄ | 0.5915 | | TPC-H 20 || 410.2 | 402.1 | -2%˄ || 2.44 | 2.49 | +2%˄ | 0.0000 | | TPC-H 21 || 5414.3 | 5312.0 | -2% || 0.18 | 0.19 | +2% | 0.1280 | | TPC-H 22 || 476.5 | 473.3 | -1%˄ || 2.10 | 2.11 | +1%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 40046.7 | 38955.4 | -3% || | | | | | Geomean || | | || | | +3% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - single-threaded, SF 0.01

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st_s01.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st_s01.json | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 12:04:22 | 2022-07-23 11:30:30 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 0.009999999776482582 | 0.009999999776482582 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 4.8 | 4.9 | +0%˄ || 206.60 | 205.96 | -0%˄ | 0.3762 | | TPC-H 02 || 3.8 | 3.8 | -0%˄ || 263.16 | 263.65 | +0%˄ | 0.9750 | +| TPC-H 03 || 0.7 | 0.7 | -7%˄ || 1386.19 | 1482.76 | +7%˄ | 0.0000 | +| TPC-H 04 || 0.5 | 0.4 | -6%˄ || 2133.89 | 2271.11 | +6%˄ | 0.0000 | | TPC-H 05 || 1.0 | 1.0 | -2%˄ || 971.81 | 994.44 | +2%˄ | 0.0013 | | TPC-H 06 || 0.2 | 0.2 | +1%˄ || 5714.13 | 5635.68 | -1%˄ | 0.0013 | | TPC-H 07 || 10.9 | 10.8 | -1%˄ || 91.86 | 92.87 | +1%˄ | 0.8686 | | TPC-H 08 || 16.0 | 15.7 | -2%˄ || 62.53 | 63.58 | +2%˄ | 0.0894 | -| TPC-H 09 || 3.1 | 3.9 | +25%˄ || 323.53 | 257.94 | -20%˄ | 0.1298 | | TPC-H 10 || 0.8 | 0.8 | -3%˄ || 1223.48 | 1266.15 | +3%˄ | 0.0000 | | TPC-H 11 || 0.2 | 0.2 | +0%˄ || 4718.01 | 4709.33 | -0%˄ | 0.8281 | | TPC-H 12 || 0.6 | 0.6 | +0%˄ || 1632.06 | 1628.97 | -0%˄ | 0.5216 | -| TPC-H 13 || 1.9 | 2.1 | +7%˄ || 520.62 | 485.24 | -7%˄ | 0.0000 | | TPC-H 14 || 0.3 | 0.3 | +3%˄ || 2944.67 | 2862.08 | -3%˄ | 0.0026 | | TPC-H 15 || 1.1 | 1.1 | +1%˄ || 886.39 | 877.85 | -1%˄ | 0.0017 | | TPC-H 16 || 2.0 | 1.9 | -0%˄ || 510.59 | 512.95 | +0%˄ | 0.0941 | -| TPC-H 17 || 0.6 | 0.7 | +6%˄ || 1592.72 | 1509.60 | -5%˄ | 0.6539 | | TPC-H 18 || 1.2 | 1.1 | -1%˄ || 866.72 | 877.74 | +1%˄ | 0.0000 | | TPC-H 19 || 4.9 | 4.8 | -3%˄ || 203.62 | 208.90 | +3%˄ | 0.0000 | | TPC-H 20 || 2.1 | 2.1 | +2%˄ || 482.12 | 473.33 | -2%˄ | 0.3707 | +| TPC-H 21 || 1.5 | 1.4 | -9%˄ || 647.24 | 710.39 | +10%˄ | 0.0339 | | TPC-H 22 || 1.0 | 1.0 | -1%˄ || 1001.69 | 1009.69 | +1%˄ | 0.0003 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 59.3 | 59.5 | +0% || | | | | | Geomean || | | || | | -0% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 8 cores, SF 10.0

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt_ordered.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt_ordered.json | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 12:04:50 | 2022-07-23 11:31:01 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | TPC-H 01 || 4806.9 | 4788.3 | -0% || 0.20 | 0.20 | +0% | (run time too short) | | TPC-H 02 || 64.9 | 63.2 | -3% || 14.16 | 14.50 | +2% | 0.0123 | | TPC-H 03 || 1212.3 | 1194.3 | -1% || 0.82 | 0.82 | +0% | (run time too short) | | TPC-H 04 || 670.6 | 655.8 | -2% || 1.47 | 1.50 | +2% | 0.0748 | | TPC-H 05 || 1073.1 | 1071.1 | -0% || 0.92 | 0.92 | -0% | (run time too short) | | TPC-H 06 || 72.2 | 71.8 | -0% || 12.92 | 13.02 | +1% | 0.2259 | | TPC-H 07 || 483.0 | 474.7 | -2% || 2.05 | 2.08 | +2% | 0.0017 | | TPC-H 08 || 440.8 | 431.3 | -2% || 2.23 | 2.28 | +2% | 0.0001 | | TPC-H 09 || 2525.9 | 2508.9 | -1% || 0.38 | 0.38 | -0% | (run time too short) | | TPC-H 10 || 1683.8 | 1680.6 | -0% || 0.58 | 0.58 | +0% | (run time too short) | | TPC-H 11 || 92.2 | 90.8 | -2% || 10.28 | 10.45 | +2% | 0.0080 | | TPC-H 12 || 594.5 | 585.0 | -2% || 1.67 | 1.68 | +1% | 0.0001 | | TPC-H 13 || 3081.7 | 3065.1 | -1% || 0.32 | 0.32 | -0% | (run time too short) | | TPC-H 14 || 174.4 | 173.4 | -1% || 5.57 | 5.60 | +1% | 0.1465 | | TPC-H 15 || 167.8 | 167.7 | -0% || 5.78 | 5.78 | -0% | 0.8777 | | TPC-H 16 || 772.1 | 785.6 | +2% || 1.28 | 1.25 | -3% | (run time too short) | | TPC-H 17 || 94.9 | 95.0 | +0% || 10.00 | 10.00 | +0% | 0.7747 | | TPC-H 18 || 2515.9 | 2585.1 | +3% || 0.38 | 0.38 | +0% | (run time too short) | | TPC-H 19 || 157.5 | 156.1 | -1% || 6.15 | 6.20 | +1% | 0.1761 | | TPC-H 20 || 252.1 | 250.4 | -1% || 3.88 | 3.92 | +1% | 0.3152 | | TPC-H 21 || 1333.5 | 1327.0 | -0% || 0.73 | 0.75 | +2% | (run time too short) | | TPC-H 22 || 174.0 | 173.7 | -0% || 5.57 | 5.58 | +0% | 0.7407 | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 22444.2 | 22394.8 | -0% || | | | | | Geomean || | | || | | +1% | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkTPCH - multi-threaded, shuffled, 8 clients, 8 cores, SF 10.0

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 12:28:39 | 2022-07-23 11:54:49 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 5692.0 | 5725.6 | +1% || 0.42 | 0.42 | +1% | 0.6996 | +| TPC-H 02 || 487.0 | 369.3 | -24% || 0.42 | 0.43 | +1% | 0.1882 | +| TPC-H 03 || 4063.8 | 3607.1 | -11% || 0.42 | 0.43 | +1% | 0.0173 | -| TPC-H 04 || 2407.5 | 2657.5 | +10% || 0.42 | 0.43 | +1% | 0.1708 | | TPC-H 05 || 4430.4 | 4489.5 | +1% || 0.42 | 0.43 | +1% | 0.8036 | | TPC-H 06 || 515.3 | 492.4 | -4% || 0.42 | 0.43 | +1% | 0.7015 | | TPC-H 07 || 2725.1 | 2744.6 | +1% || 0.42 | 0.43 | +1% | 0.9182 | | TPC-H 08 || 2278.2 | 2347.3 | +3% || 0.42 | 0.42 | +1% | 0.7225 | | TPC-H 09 || 6373.1 | 6650.6 | +4% || 0.42 | 0.42 | +1% | 0.2708 | | TPC-H 10 || 5295.6 | 5275.2 | -0% || 0.42 | 0.43 | +1% | 0.9372 | -| TPC-H 11 || 472.0 | 522.0 | +11% || 0.42 | 0.43 | +1% | 0.5203 | | TPC-H 12 || 2426.7 | 2434.1 | +0% || 0.42 | 0.43 | +1% | 0.9677 | | TPC-H 13 || 6212.7 | 6221.5 | +0% || 0.42 | 0.42 | +1% | 0.9558 | +| TPC-H 14 || 1188.3 | 1048.7 | -12% || 0.42 | 0.43 | +1% | 0.2369 | | TPC-H 15 || 461.3 | 462.4 | +0% || 0.42 | 0.43 | +1% | 0.9826 | | TPC-H 16 || 2419.4 | 2397.6 | -1% || 0.42 | 0.42 | +1% | 0.9104 | +| TPC-H 17 || 705.3 | 603.7 | -14% || 0.42 | 0.43 | +1% | 0.2603 | | TPC-H 18 || 3690.7 | 3622.1 | -2% || 0.42 | 0.42 | +1% | 0.5738 | +| TPC-H 19 || 932.8 | 813.4 | -13% || 0.42 | 0.43 | +1% | 0.3276 | -| TPC-H 20 || 1222.7 | 1325.4 | +8% || 0.42 | 0.43 | +1% | 0.3791 | +| TPC-H 21 || 6662.3 | 6078.0 | -9% || 0.42 | 0.42 | +1% | 0.0597 | +| TPC-H 22 || 1008.4 | 912.6 | -10% || 0.42 | 0.43 | +1% | 0.3486 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 61670.7 | 60800.5 | -1% || | | | | | Geomean || | | || | | +1% | | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - single-threaded

Sum of avg. item runtimes: -6% || Geometric mean of throughput changes: +13%

Configuration Overview - click to expand

```diff +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 12:50:25 | 2022-07-23 12:16:33 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 265.0 | 262.0 | -1%˄ || 3.77 | 3.82 | +1%˄ | 0.0083 | +| 03 || 80.0 | 67.3 | -16%˄ || 12.50 | 14.86 | +19%˄ | 0.0000 | +| 06 || 169.7 | 149.3 | -12%˄ || 5.89 | 6.70 | +14%˄ | 0.0000 | +| 07 || 314.6 | 263.9 | -16%˄ || 3.18 | 3.79 | +19%˄ | 0.0000 | | 09 || 635.6 | 637.8 | +0% || 1.57 | 1.57 | -0% | 0.0000 | +| 10 || 163.3 | 119.6 | -27%˄ || 6.12 | 8.36 | +37%˄ | 0.0000 | +| 13 || 445.1 | 396.7 | -11%˄ || 2.25 | 2.52 | +12%˄ | 0.0000 | +| 15 || 114.9 | 106.4 | -7%˄ || 8.70 | 9.40 | +8%˄ | 0.0000 | +| 16 || 159.4 | 139.7 | -12%˄ || 6.27 | 7.16 | +14%˄ | 0.0000 | +| 17 || 337.8 | 302.6 | -10%˄ || 2.96 | 3.30 | +12%˄ | 0.0000 | +| 19 || 118.5 | 96.0 | -19%˄ || 8.44 | 10.42 | +23%˄ | 0.0000 | +| 25 || 191.2 | 158.9 | -17%˄ || 5.23 | 6.29 | +20%˄ | 0.0000 | +| 26 || 136.1 | 125.5 | -8%˄ || 7.35 | 7.97 | +8%˄ | 0.0000 | | 28 || 609.3 | 597.8 | -2%˄ || 1.64 | 1.67 | +2%˄ | 0.0000 | +| 29 || 504.6 | 467.2 | -7%˄ || 1.98 | 2.14 | +8%˄ | 0.0000 | +| 31 || 1333.1 | 1256.0 | -6% || 0.75 | 0.80 | +6% | 0.0000 | +| 32 || 51.0 | 42.4 | -17%˄ || 19.59 | 23.57 | +20%˄ | 0.0000 | +| 34 || 180.8 | 147.2 | -19%˄ || 5.53 | 6.79 | +23%˄ | 0.0000 | +| 35 || 676.2 | 628.3 | -7% || 1.48 | 1.59 | +8% | 0.0000 | +| 37 || 328.3 | 234.1 | -29%˄ || 3.05 | 4.27 | +40%˄ | 0.0000 | +| 39a || 1801.3 | 1655.4 | -8% || 0.56 | 0.60 | +9% | 0.0000 | +| 39b || 1827.3 | 1655.5 | -9% || 0.55 | 0.60 | +10% | 0.0000 | | 41 || 286.8 | 289.1 | +1%˄ || 3.49 | 3.46 | -1%˄ | 0.0000 | +| 42 || 99.0 | 77.3 | -22%˄ || 10.10 | 12.94 | +28%˄ | 0.0000 | +| 43 || 961.9 | 911.6 | -5% || 1.04 | 1.10 | +6% | 0.0000 | +| 45 || 122.5 | 117.3 | -4%˄ || 8.16 | 8.53 | +5%˄ | 0.0000 | +| 48 || 1047.7 | 988.0 | -6% || 0.95 | 1.01 | +6% | 0.0000 | +| 50 || 129.5 | 109.5 | -16%˄ || 7.72 | 9.13 | +18%˄ | 0.0000 | +| 52 || 97.6 | 78.0 | -20%˄ || 10.24 | 12.81 | +25%˄ | 0.0000 | +| 55 || 94.1 | 74.8 | -21%˄ || 10.63 | 13.37 | +26%˄ | 0.0000 | | 62 || 532.8 | 524.1 | -2%˄ || 1.88 | 1.91 | +2%˄ | 0.0000 | | 65 || 1777.2 | 1718.7 | -3% || 0.56 | 0.58 | +3% | 0.0000 | +| 69 || 151.5 | 119.3 | -21%˄ || 6.60 | 8.38 | +27%˄ | 0.0000 | +| 73 || 95.0 | 75.4 | -21%˄ || 10.53 | 13.26 | +26%˄ | 0.0000 | | 79 || 483.8 | 463.9 | -4%˄ || 2.07 | 2.16 | +4%˄ | 0.0000 | | 81 || 210.0 | 202.2 | -4%˄ || 4.76 | 4.95 | +4%˄ | 0.0000 | +| 82 || 431.9 | 328.8 | -24%˄ || 2.32 | 3.04 | +31%˄ | 0.0000 | +| 83 || 43.9 | 39.4 | -10%˄ || 22.76 | 25.35 | +11%˄ | 0.0000 | +| 85 || 133.3 | 124.7 | -6%˄ || 7.50 | 8.02 | +7%˄ | 0.0000 | +| 88 || 820.6 | 657.2 | -20% || 1.22 | 1.52 | +25% | 0.0000 | +| 91 || 18.0 | 16.2 | -10%˄ || 55.67 | 61.56 | +11%˄ | 0.0000 | +| 92 || 40.8 | 35.8 | -12%˄ || 24.52 | 27.95 | +14%˄ | 0.0000 | | 93 || 3974.1 | 3827.5 | -4% || 0.25 | 0.26 | +4% | 0.0000 | +| 94 || 106.2 | 95.0 | -11%˄ || 9.41 | 10.53 | +12%˄ | 0.0000 | | 95 || 9229.1 | 8951.6 | -3% || 0.11 | 0.11 | +3% | ˅ | +| 96 || 76.0 | 57.7 | -24%˄ || 13.15 | 17.33 | +32%˄ | 0.0000 | | 97 || 3291.9 | 3269.7 | -1% || 0.30 | 0.31 | +1% | 0.0883 | | 99 || 1020.9 | 1019.3 | -0% || 0.98 | 0.98 | +0% | 0.2497 | +---------++----------+---------+--------++----------+----------+--------+---------+ +| Sum || 35719.1 | 33681.5 | -6% || | | | | +| Geomean || | | || | | +13% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | | || ˅ Insufficient number of runs for p-value calculation | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - multi-threaded, shuffled, 8 clients, 8 cores

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 13:17:28 | 2022-07-23 12:44:27 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ -| 01 || 616.8 | 653.1 | +6% || 0.46 | 0.47 | +2% | 0.6006 | +| 03 || 278.1 | 185.4 | -33% || 0.46 | 0.47 | +2% | 0.0416 | -| 06 || 582.5 | 654.4 | +12% || 0.46 | 0.47 | +1% | 0.3317 | -| 07 || 734.6 | 843.3 | +15% || 0.46 | 0.47 | +1% | 0.1934 | | 09 || 1343.2 | 1286.6 | -4% || 0.46 | 0.47 | +1% | 0.4440 | -| 10 || 388.3 | 463.7 | +19% || 0.46 | 0.47 | +1% | 0.2744 | | 13 || 1453.1 | 1506.3 | +4% || 0.46 | 0.47 | +1% | 0.6651 | +| 15 || 504.7 | 461.6 | -9% || 0.46 | 0.47 | +1% | 0.5780 | | 16 || 778.1 | 753.5 | -3% || 0.46 | 0.47 | +1% | 0.8167 | +| 17 || 1116.2 | 1001.2 | -10% || 0.46 | 0.47 | +2% | 0.2794 | -| 19 || 446.5 | 492.0 | +10% || 0.46 | 0.47 | +1% | 0.4701 | +| 25 || 839.7 | 742.4 | -12% || 0.46 | 0.47 | +2% | 0.3422 | -| 26 || 423.4 | 527.6 | +25% || 0.46 | 0.47 | +2% | 0.1063 | -| 28 || 500.2 | 537.2 | +7% || 0.46 | 0.47 | +1% | 0.5167 | +| 29 || 1432.1 | 1305.0 | -9% || 0.46 | 0.47 | +2% | 0.2080 | | 31 || 2054.8 | 2008.8 | -2% || 0.46 | 0.47 | +2% | 0.7746 | +| 32 || 385.2 | 290.7 | -25% || 0.46 | 0.47 | +2% | 0.1982 | | 34 || 423.6 | 417.2 | -1% || 0.46 | 0.47 | +2% | 0.9031 | -| 35 || 1727.1 | 2024.7 | +17% || 0.46 | 0.47 | +1% | 0.0309 | -| 37 || 516.7 | 589.3 | +14% || 0.46 | 0.47 | +1% | 0.3158 | +| 39a || 2312.2 | 2206.6 | -5% || 0.46 | 0.47 | +1% | 0.4627 | | 39b || 2075.7 | 2132.4 | +3% || 0.46 | 0.47 | +2% | 0.5964 | +| 41 || 2921.8 | 2760.2 | -6% || 0.46 | 0.47 | +2% | 0.2894 | -| 42 || 267.3 | 331.1 | +24% || 0.46 | 0.47 | +1% | 0.1622 | | 43 || 1344.7 | 1378.3 | +2% || 0.46 | 0.47 | +1% | 0.7271 | +| 45 || 619.1 | 506.1 | -18% || 0.46 | 0.47 | +1% | 0.1526 | | 48 || 2704.2 | 2699.3 | -0% || 0.46 | 0.47 | +2% | 0.9770 | +| 50 || 582.3 | 504.3 | -13% || 0.46 | 0.47 | +1% | 0.2943 | | 52 || 287.7 | 293.9 | +2% || 0.46 | 0.47 | +2% | 0.9056 | | 55 || 282.7 | 275.5 | -3% || 0.46 | 0.47 | +2% | 0.8991 | | 62 || 789.9 | 789.9 | -0% || 0.46 | 0.47 | +1% | 0.9993 | | 65 || 3556.1 | 3621.8 | +2% || 0.46 | 0.47 | +1% | 0.3770 | | 69 || 641.7 | 627.2 | -2% || 0.46 | 0.47 | +1% | 0.8696 | +| 73 || 339.9 | 273.7 | -19% || 0.46 | 0.47 | +1% | 0.1984 | | 79 || 1137.5 | 1183.9 | +4% || 0.46 | 0.47 | +1% | 0.5813 | -| 81 || 804.2 | 880.8 | +10% || 0.46 | 0.47 | +1% | 0.3954 | +| 82 || 881.1 | 705.2 | -20% || 0.46 | 0.47 | +2% | 0.0374 | +| 83 || 336.0 | 252.3 | -25% || 0.46 | 0.47 | +1% | 0.1648 | -| 85 || 659.3 | 692.4 | +5% || 0.46 | 0.47 | +1% | 0.7104 | -| 88 || 787.6 | 837.6 | +6% || 0.46 | 0.47 | +2% | 0.5863 | +| 91 || 211.4 | 155.5 | -26% || 0.46 | 0.47 | +2% | 0.2275 | +| 92 || 346.3 | 245.4 | -29% || 0.46 | 0.47 | +1% | 0.1014 | +| 93 || 2472.3 | 2333.9 | -6% || 0.46 | 0.47 | +1% | 0.3116 | -| 94 || 605.4 | 688.4 | +14% || 0.46 | 0.47 | +2% | 0.4101 | | 95 || 7831.1 | 7781.7 | -1% || 0.46 | 0.47 | +2% | 0.7502 | +| 96 || 204.8 | 171.3 | -16% || 0.46 | 0.47 | +1% | 0.3053 | | 97 || 4321.3 | 4351.0 | +1% || 0.46 | 0.47 | +1% | 0.8457 | +| 99 || 1366.4 | 1227.9 | -10% || 0.46 | 0.47 | +1% | 0.0889 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 57235.0 | 56651.4 | -1% || | | | | | Geomean || | | || | | +1% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - single-threaded

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 13:37:55 | 2022-07-23 13:04:57 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 22.7 | 22.8 | +0% || 4.30 | 4.28 | -0% | 0.2422 | | New-Order || 16.2 | 16.3 | +0% || 48.36 | 48.19 | -0% | 0.6367 | | Order-Status || 1.1 | 1.1 | +1% || 4.30 | 4.30 | +0% | 0.3446 | | Payment || 2.1 | 2.1 | +0% || 46.23 | 46.03 | -0% | 0.4274 | | Stock-Level || 3.6 | 3.5 | -1% || 4.28 | 4.27 | -0% | 0.2026 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 45.7 | 45.8 | +0% || | | | | | Geomean || | | || | | -0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - multi-threaded, shuffled, 8 clients, 8 cores

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 13:38:55 | 2022-07-23 13:05:59 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 220.1 | 219.8 | -0% || 4.31 | 4.31 | +0% | 0.8398 | | unsucc.: || 4.3 | 4.3 | +0% || 62.89 | 62.94 | +0% | | | New-Order || 57.4 | 57.2 | -0% || 134.52 | 133.68 | -1% | 0.0630 | | unsucc.: || 4.9 | 5.0 | +3% || 621.40 | 622.86 | +0% | | | Order-Status || 7.6 | 7.6 | -0% || 67.19 | 67.25 | +0% | 0.4682 | | Payment || 9.8 | 9.9 | +1% || 21.42 | 21.50 | +0% | 0.3043 | | unsucc.: || 4.4 | 4.4 | +1% || 700.91 | 701.43 | +0% | | | Stock-Level || 17.5 | 17.3 | -1% || 67.19 | 67.25 | +0% | 0.0045 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 312.5 | 311.8 | -0% || | | | | | Geomean || | | || | | -0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - single-threaded

Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +12%

Configuration Overview - click to expand

```diff +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 13:59:03 | 2022-07-23 13:27:39 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ +| 10a || 179.3 | 164.0 | -9%˄ || 5.58 | 6.10 | +9%˄ | 0.0000 | +| 10b || 128.7 | 115.3 | -10%˄ || 7.77 | 8.67 | +12%˄ | 0.0000 | -| 10c || 317.0 | 351.6 | +11%˄ || 3.15 | 2.84 | -10%˄ | 0.0000 | +| 11a || 20.0 | 15.2 | -24%˄ || 49.96 | 65.78 | +32%˄ | 0.0000 | +| 11b || 19.9 | 15.1 | -24%˄ || 50.35 | 66.19 | +31%˄ | 0.0000 | +| 11c || 25.3 | 20.4 | -19%˄ || 39.54 | 48.90 | +24%˄ | 0.0000 | +| 11d || 28.1 | 23.2 | -17%˄ || 35.63 | 43.17 | +21%˄ | 0.0000 | +| 12a || 50.3 | 47.3 | -6%˄ || 19.87 | 21.16 | +6%˄ | 0.0000 | +| 12b || 40.6 | 29.3 | -28%˄ || 24.65 | 34.12 | +38%˄ | 0.0000 | | 12c || 88.0 | 84.8 | -4%˄ || 11.37 | 11.79 | +4%˄ | 0.0000 | +| 13a || 288.0 | 271.9 | -6%˄ || 3.47 | 3.68 | +6%˄ | 0.0000 | +| 13b || 171.1 | 154.4 | -10%˄ || 5.85 | 6.48 | +11%˄ | 0.0000 | +| 13c || 133.2 | 118.7 | -11%˄ || 7.51 | 8.42 | +12%˄ | 0.0000 | | 13d || 577.5 | 558.8 | -3%˄ || 1.73 | 1.79 | +3%˄ | 0.0000 | | 14a || 431.6 | 421.3 | -2%˄ || 2.32 | 2.37 | +2%˄ | 0.0000 | | 14b || 254.0 | 249.0 | -2%˄ || 3.94 | 4.02 | +2%˄ | 0.0000 | | 14c || 503.4 | 496.5 | -1%˄ || 1.99 | 2.01 | +1%˄ | 0.0000 | +| 15a || 89.7 | 83.5 | -7%˄ || 11.14 | 11.97 | +7%˄ | 0.0000 | +| 15b || 88.8 | 82.9 | -7%˄ || 11.27 | 12.06 | +7%˄ | 0.0000 | +| 15c || 93.4 | 88.2 | -6%˄ || 10.71 | 11.34 | +6%˄ | 0.0000 | +| 15d || 91.0 | 85.0 | -7%˄ || 10.99 | 11.76 | +7%˄ | 0.0000 | | 16a || 2337.0 | 2310.9 | -1% || 0.43 | 0.43 | +1% | 0.0121 | | 16b || 3363.1 | 3336.6 | -1% || 0.30 | 0.30 | +1% | 0.1070 | | 16c || 2438.2 | 2405.2 | -1% || 0.41 | 0.42 | +1% | 0.0001 | | 16d || 2401.8 | 2378.8 | -1% || 0.42 | 0.42 | +1% | 0.0109 | | 17a || 623.2 | 601.3 | -4%˄ || 1.60 | 1.66 | +4%˄ | 0.0000 | +| 17b || 501.6 | 477.0 | -5%˄ || 1.99 | 2.10 | +5%˄ | 0.0000 | +| 17c || 480.7 | 455.6 | -5%˄ || 2.08 | 2.19 | +5%˄ | 0.0000 | +| 17d || 562.7 | 536.4 | -5%˄ || 1.78 | 1.86 | +5%˄ | 0.0000 | | 17e || 1628.4 | 1599.2 | -2% || 0.61 | 0.63 | +2% | 0.0000 | | 17f || 991.8 | 965.0 | -3% || 1.01 | 1.04 | +3% | 0.0000 | | 18a || 646.0 | 636.2 | -2% || 1.55 | 1.57 | +2% | 0.0000 | +| 18b || 124.7 | 110.7 | -11%˄ || 8.02 | 9.03 | +13%˄ | 0.0000 | | 18c || 589.1 | 569.5 | -3%˄ || 1.70 | 1.76 | +3%˄ | 0.0000 | +| 19a || 194.7 | 185.6 | -5%˄ || 5.14 | 5.39 | +5%˄ | 0.0000 | +| 19b || 137.4 | 126.3 | -8%˄ || 7.28 | 7.92 | +9%˄ | 0.0000 | | 19c || 206.7 | 197.8 | -4%˄ || 4.84 | 5.05 | +4%˄ | 0.0000 | | 19d || 738.1 | 727.7 | -1% || 1.35 | 1.37 | +1% | 0.0000 | +| 1a || 11.9 | 9.0 | -25%˄ || 83.79 | 111.44 | +33%˄ | 0.0000 | +| 1b || 11.4 | 8.4 | -26%˄ || 88.02 | 118.96 | +35%˄ | 0.0000 | +| 1c || 12.5 | 9.5 | -23%˄ || 80.29 | 104.72 | +30%˄ | 0.0000 | +| 1d || 11.4 | 8.4 | -26%˄ || 87.94 | 118.36 | +35%˄ | 0.0000 | +| 20a || 407.0 | 387.4 | -5%˄ || 2.46 | 2.58 | +5%˄ | 0.0000 | +| 20b || 245.8 | 226.6 | -8%˄ || 4.07 | 4.41 | +8%˄ | 0.0000 | +| 20c || 237.5 | 218.7 | -8%˄ || 4.21 | 4.57 | +9%˄ | 0.0000 | +| 21a || 48.6 | 35.4 | -27%˄ || 20.59 | 28.23 | +37%˄ | 0.0000 | +| 21b || 24.2 | 19.2 | -21%˄ || 41.33 | 52.10 | +26%˄ | 0.0000 | +| 21c || 48.1 | 35.0 | -27%˄ || 20.81 | 28.59 | +37%˄ | 0.0000 | | 22a || 216.8 | 209.4 | -3%˄ || 4.61 | 4.77 | +4%˄ | 0.0000 | +| 22b || 156.8 | 149.8 | -4%˄ || 6.38 | 6.67 | +5%˄ | 0.0000 | | 22c || 517.9 | 507.1 | -2%˄ || 1.93 | 1.97 | +2%˄ | 0.0000 | | 22d || 947.5 | 912.7 | -4% || 1.06 | 1.10 | +4% | 0.0000 | +| 23a || 54.9 | 42.8 | -22%˄ || 18.21 | 23.38 | +28%˄ | 0.0000 | +| 23b || 55.5 | 44.0 | -21%˄ || 18.01 | 22.75 | +26%˄ | 0.0000 | +| 23c || 94.0 | 84.6 | -10%˄ || 10.63 | 11.82 | +11%˄ | 0.0000 | +| 24a || 169.6 | 144.0 | -15%˄ || 5.90 | 6.95 | +18%˄ | 0.0000 | +| 24b || 148.4 | 123.1 | -17%˄ || 6.74 | 8.12 | +21%˄ | 0.0000 | +| 25a || 292.9 | 279.2 | -5%˄ || 3.41 | 3.58 | +5%˄ | 0.0000 | +| 25b || 136.9 | 113.1 | -17%˄ || 7.30 | 8.84 | +21%˄ | 0.0000 | | 25c || 945.2 | 922.4 | -2% || 1.06 | 1.08 | +2% | 0.0000 | +| 26a || 180.8 | 160.7 | -11%˄ || 5.53 | 6.22 | +13%˄ | 0.0000 | +| 26b || 148.4 | 129.2 | -13%˄ || 6.74 | 7.74 | +15%˄ | 0.0000 | +| 26c || 249.7 | 230.6 | -8%˄ || 4.01 | 4.34 | +8%˄ | 0.0000 | +| 27a || 27.9 | 23.0 | -18%˄ || 35.80 | 43.49 | +21%˄ | 0.0000 | +| 27b || 27.4 | 22.5 | -18%˄ || 36.55 | 44.52 | +22%˄ | 0.0000 | +| 27c || 48.2 | 35.1 | -27%˄ || 20.73 | 28.45 | +37%˄ | 0.0000 | | 28a || 306.5 | 298.9 | -3%˄ || 3.26 | 3.35 | +3%˄ | 0.0000 | +| 28b || 55.4 | 49.5 | -11%˄ || 18.05 | 20.18 | +12%˄ | 0.0000 | | 28c || 347.2 | 343.8 | -1%˄ || 2.88 | 2.91 | +1%˄ | 0.0000 | +| 29a || 133.3 | 102.4 | -23%˄ || 7.50 | 9.76 | +30%˄ | 0.0000 | +| 29b || 106.6 | 76.8 | -28%˄ || 9.38 | 13.03 | +39%˄ | 0.0000 | +| 29c || 176.2 | 148.3 | -16%˄ || 5.68 | 6.74 | +19%˄ | 0.0000 | +| 2a || 51.2 | 46.7 | -9%˄ || 19.53 | 21.42 | +10%˄ | 0.0000 | +| 2b || 42.5 | 38.3 | -10%˄ || 23.51 | 26.11 | +11%˄ | 0.0000 | +| 2c || 27.0 | 22.3 | -18%˄ || 37.02 | 44.88 | +21%˄ | 0.0000 | +| 2d || 70.2 | 66.8 | -5%˄ || 14.23 | 14.98 | +5%˄ | 0.0000 | +| 30a || 166.2 | 148.5 | -11%˄ || 6.02 | 6.73 | +12%˄ | 0.0000 | +| 30b || 144.4 | 126.4 | -12%˄ || 6.92 | 7.91 | +14%˄ | 0.0000 | +| 30c || 404.3 | 386.7 | -4%˄ || 2.47 | 2.59 | +5%˄ | 0.0000 | +| 31a || 172.4 | 146.0 | -15%˄ || 5.80 | 6.85 | +18%˄ | 0.0000 | +| 31b || 150.9 | 124.4 | -18%˄ || 6.63 | 8.04 | +21%˄ | 0.0000 | +| 31c || 197.0 | 168.6 | -14%˄ || 5.08 | 5.93 | +17%˄ | 0.0000 | +| 32a || 17.1 | 12.9 | -24%˄ || 58.64 | 77.26 | +32%˄ | 0.0000 | +| 32b || 36.9 | 32.9 | -11%˄ || 27.10 | 30.42 | +12%˄ | 0.0000 | +| 33a || 26.0 | 19.4 | -25%˄ || 38.43 | 51.51 | +34%˄ | 0.0000 | +| 33b || 25.5 | 18.7 | -27%˄ || 39.24 | 53.43 | +36%˄ | 0.0000 | +| 33c || 29.3 | 22.6 | -23%˄ || 34.14 | 44.31 | +30%˄ | 0.0000 | | 3a || 127.5 | 122.9 | -4%˄ || 7.84 | 8.14 | +4%˄ | 0.0000 | +| 3b || 16.3 | 12.6 | -23%˄ || 61.37 | 79.25 | +29%˄ | 0.0000 | | 3c || 458.5 | 449.8 | -2%˄ || 2.18 | 2.22 | +2%˄ | 0.0000 | | 4a || 107.0 | 104.4 | -2%˄ || 9.35 | 9.58 | +2%˄ | 0.0000 | +| 4b || 15.3 | 11.4 | -25%˄ || 65.55 | 87.38 | +33%˄ | 0.0000 | | 4c || 123.2 | 120.6 | -2%˄ || 8.11 | 8.29 | +2%˄ | 0.0000 | | 5a || 54.2 | 53.7 | -1%˄ || 18.43 | 18.62 | +1%˄ | 0.0000 | -| 5b || 125.4 | 134.4 | +7%˄ || 7.97 | 7.44 | -7%˄ | 0.0000 | | 5c || 194.3 | 198.2 | +2%˄ || 5.15 | 5.04 | -2%˄ | 0.0000 | +| 6a || 119.3 | 102.5 | -14%˄ || 8.38 | 9.75 | +16%˄ | 0.0000 | +| 6b || 130.6 | 114.4 | -12%˄ || 7.65 | 8.74 | +14%˄ | 0.0000 | +| 6c || 114.1 | 98.1 | -14%˄ || 8.77 | 10.19 | +16%˄ | 0.0000 | | 6d || 558.0 | 535.1 | -4%˄ || 1.79 | 1.87 | +4%˄ | 0.0000 | +| 6e || 119.1 | 102.7 | -14%˄ || 8.40 | 9.74 | +16%˄ | 0.0000 | | 6f || 720.6 | 706.2 | -2% || 1.39 | 1.42 | +2% | 0.0000 | +| 7a || 78.2 | 53.8 | -31%˄ || 12.79 | 18.60 | +45%˄ | 0.0000 | +| 7b || 74.8 | 50.1 | -33%˄ || 13.38 | 19.96 | +49%˄ | 0.0000 | +| 7c || 601.2 | 565.2 | -6%˄ || 1.66 | 1.77 | +6%˄ | 0.0000 | +| 8a || 43.2 | 39.1 | -10%˄ || 23.14 | 25.58 | +11%˄ | 0.0000 | +| 8b || 40.9 | 37.2 | -9%˄ || 24.44 | 26.91 | +10%˄ | 0.0000 | | 8c || 1955.7 | 1940.3 | -1% || 0.51 | 0.52 | +1% | 0.0000 | +| 8d || 347.6 | 328.6 | -5%˄ || 2.88 | 3.04 | +6%˄ | 0.0000 | | 9a || 193.2 | 193.3 | +0%˄ || 5.18 | 5.17 | -0%˄ | 0.5512 | | 9b || 109.0 | 106.6 | -2%˄ || 9.17 | 9.38 | +2%˄ | 0.0000 | | 9c || 274.1 | 274.1 | -0%˄ || 3.65 | 3.65 | +0%˄ | 0.7176 | | 9d || 479.8 | 481.6 | +0%˄ || 2.08 | 2.08 | -0%˄ | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 36928.1 | 35525.0 | -4% || | | | | +| Geomean || | | || | | +12% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 8 clients, 8 cores

Sum of avg. item runtimes: -2% || Geometric mean of throughput changes: +3%

Configuration Overview - click to expand

```diff +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | /Users/mrcl/projects/hyrise/build-rel-clang-12/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-24 14:43:25 | 2022-07-23 14:10:35 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ +| 10a || 231.1 | 196.0 | -15% || 0.47 | 0.49 | +3% | 0.3332 | +| 10b || 137.5 | 128.4 | -7% || 0.47 | 0.49 | +3% | 0.5763 | +| 10c || 456.9 | 419.9 | -8% || 0.47 | 0.49 | +3% | 0.3298 | +| 11a || 60.4 | 43.6 | -28% || 0.47 | 0.49 | +3% | 0.2214 | +| 11b || 49.8 | 45.7 | -8% || 0.47 | 0.49 | +3% | 0.7041 | +| 11c || 72.9 | 57.8 | -21% || 0.47 | 0.49 | +3% | 0.0979 | +| 11d || 102.6 | 84.4 | -18% || 0.47 | 0.49 | +3% | 0.3599 | +| 12a || 192.4 | 168.9 | -12% || 0.47 | 0.49 | +3% | 0.3291 | +| 12b || 87.3 | 64.9 | -26% || 0.47 | 0.49 | +3% | 0.2209 | +| 12c || 336.5 | 299.1 | -11% || 0.47 | 0.49 | +3% | 0.2283 | -| 13a || 588.7 | 665.4 | +13% || 0.47 | 0.49 | +3% | 0.0888 | | 13b || 301.4 | 310.6 | +3% || 0.47 | 0.49 | +3% | 0.8058 | +| 13c || 262.3 | 207.0 | -21% || 0.47 | 0.49 | +3% | 0.0552 | | 13d || 993.4 | 1010.5 | +2% || 0.47 | 0.49 | +3% | 0.7434 | -| 14a || 762.3 | 849.7 | +11% || 0.47 | 0.49 | +3% | 0.0898 | +| 14b || 516.9 | 467.3 | -10% || 0.47 | 0.49 | +3% | 0.1961 | +| 14c || 990.2 | 909.7 | -8% || 0.47 | 0.49 | +3% | 0.1061 | | 15a || 164.6 | 163.4 | -1% || 0.47 | 0.49 | +3% | 0.9541 | -| 15b || 135.7 | 144.7 | +7% || 0.47 | 0.49 | +3% | 0.6782 | | 15c || 155.2 | 150.7 | -3% || 0.47 | 0.49 | +3% | 0.8135 | -| 15d || 195.3 | 210.1 | +8% || 0.47 | 0.49 | +3% | 0.5822 | | 16a || 2706.4 | 2611.4 | -4% || 0.47 | 0.49 | +3% | 0.1346 | | 16b || 3923.8 | 4013.7 | +2% || 0.47 | 0.48 | +3% | 0.1526 | +| 16c || 2806.2 | 2647.6 | -6% || 0.47 | 0.48 | +3% | 0.0040 | | 16d || 2800.2 | 2736.3 | -2% || 0.47 | 0.49 | +3% | 0.3050 | +| 17a || 824.6 | 753.2 | -9% || 0.47 | 0.49 | +3% | 0.0947 | | 17b || 640.1 | 647.4 | +1% || 0.47 | 0.49 | +3% | 0.8754 | | 17c || 571.3 | 581.2 | +2% || 0.47 | 0.49 | +3% | 0.8326 | | 17d || 582.8 | 575.7 | -1% || 0.47 | 0.49 | +3% | 0.8556 | | 17e || 1554.2 | 1590.1 | +2% || 0.47 | 0.48 | +2% | 0.4611 | -| 17f || 1052.1 | 1151.5 | +9% || 0.47 | 0.49 | +3% | 0.0451 | | 18a || 777.0 | 749.0 | -4% || 0.47 | 0.49 | +3% | 0.5194 | | 18b || 160.2 | 166.3 | +4% || 0.47 | 0.49 | +3% | 0.7817 | -| 18c || 637.7 | 676.0 | +6% || 0.47 | 0.49 | +3% | 0.2602 | -| 19a || 459.6 | 505.3 | +10% || 0.47 | 0.49 | +3% | 0.3347 | -| 19b || 340.6 | 378.4 | +11% || 0.47 | 0.49 | +3% | 0.3154 | -| 19c || 520.0 | 549.6 | +6% || 0.47 | 0.49 | +3% | 0.5418 | | 19d || 1254.7 | 1252.7 | -0% || 0.47 | 0.49 | +3% | 0.9718 | +| 1a || 37.4 | 32.1 | -14% || 0.47 | 0.49 | +3% | 0.4611 | +| 1b || 42.1 | 36.9 | -12% || 0.47 | 0.49 | +3% | 0.6587 | -| 1c || 41.5 | 46.9 | +13% || 0.47 | 0.49 | +3% | 0.6058 | +| 1d || 33.2 | 28.8 | -13% || 0.47 | 0.49 | +3% | 0.6358 | | 20a || 599.8 | 608.8 | +2% || 0.47 | 0.49 | +3% | 0.8517 | | 20b || 422.8 | 409.2 | -3% || 0.47 | 0.49 | +3% | 0.6732 | | 20c || 397.2 | 406.8 | +2% || 0.47 | 0.49 | +3% | 0.7639 | | 21a || 98.7 | 96.2 | -3% || 0.47 | 0.49 | +3% | 0.9207 | -| 21b || 64.1 | 72.3 | +13% || 0.47 | 0.49 | +3% | 0.4945 | -| 21c || 67.6 | 73.5 | +9% || 0.47 | 0.49 | +3% | 0.6060 | +| 22a || 616.5 | 504.8 | -18% || 0.47 | 0.49 | +3% | 0.0257 | | 22b || 371.2 | 366.1 | -1% || 0.47 | 0.49 | +3% | 0.9049 | | 22c || 930.4 | 911.8 | -2% || 0.47 | 0.49 | +3% | 0.7550 | | 22d || 1321.2 | 1344.9 | +2% || 0.47 | 0.49 | +3% | 0.7214 | +| 23a || 181.6 | 127.7 | -30% || 0.47 | 0.49 | +3% | 0.0135 | | 23b || 114.2 | 116.5 | +2% || 0.47 | 0.49 | +3% | 0.9105 | -| 23c || 163.1 | 172.9 | +6% || 0.47 | 0.49 | +3% | 0.6638 | +| 24a || 263.8 | 250.2 | -5% || 0.47 | 0.49 | +3% | 0.6840 | +| 24b || 182.7 | 157.1 | -14% || 0.47 | 0.49 | +3% | 0.2990 | | 25a || 537.7 | 548.2 | +2% || 0.47 | 0.49 | +3% | 0.8301 | +| 25b || 139.1 | 128.2 | -8% || 0.47 | 0.49 | +3% | 0.5336 | | 25c || 1077.5 | 1038.5 | -4% || 0.47 | 0.49 | +3% | 0.5705 | | 26a || 305.6 | 315.6 | +3% || 0.47 | 0.49 | +3% | 0.6995 | | 26b || 200.2 | 208.1 | +4% || 0.47 | 0.49 | +3% | 0.7374 | | 26c || 469.3 | 475.2 | +1% || 0.47 | 0.49 | +3% | 0.8903 | -| 27a || 83.7 | 109.8 | +31% || 0.47 | 0.49 | +3% | 0.1574 | -| 27b || 71.4 | 104.0 | +46% || 0.47 | 0.49 | +3% | 0.0615 | -| 27c || 75.3 | 80.2 | +7% || 0.47 | 0.49 | +3% | 0.7147 | | 28a || 789.7 | 775.2 | -2% || 0.47 | 0.49 | +3% | 0.8080 | -| 28b || 173.0 | 195.7 | +13% || 0.47 | 0.49 | +3% | 0.3749 | +| 28c || 803.5 | 758.6 | -6% || 0.47 | 0.49 | +3% | 0.3805 | +| 29a || 250.5 | 206.5 | -18% || 0.47 | 0.49 | +3% | 0.1277 | +| 29b || 157.3 | 147.2 | -6% || 0.47 | 0.49 | +3% | 0.6589 | | 29c || 237.1 | 237.3 | +0% || 0.47 | 0.49 | +3% | 0.9967 | +| 2a || 181.5 | 172.5 | -5% || 0.47 | 0.49 | +3% | 0.7522 | | 2b || 139.7 | 140.0 | +0% || 0.47 | 0.49 | +3% | 0.9904 | -| 2c || 69.7 | 92.9 | +33% || 0.47 | 0.49 | +3% | 0.1417 | -| 2d || 194.4 | 243.3 | +25% || 0.47 | 0.49 | +3% | 0.0985 | | 30a || 339.7 | 326.1 | -4% || 0.47 | 0.49 | +3% | 0.7243 | +| 30b || 261.2 | 233.0 | -11% || 0.47 | 0.49 | +3% | 0.3997 | +| 30c || 794.5 | 727.8 | -8% || 0.47 | 0.49 | +3% | 0.2296 | | 31a || 328.7 | 340.0 | +3% || 0.47 | 0.49 | +3% | 0.7717 | | 31b || 161.6 | 159.3 | -1% || 0.47 | 0.49 | +3% | 0.9155 | | 31c || 349.1 | 337.6 | -3% || 0.47 | 0.49 | +3% | 0.7431 | +| 32a || 46.7 | 39.4 | -16% || 0.47 | 0.49 | +3% | 0.5401 | +| 32b || 147.9 | 130.9 | -11% || 0.47 | 0.49 | +3% | 0.4834 | +| 33a || 94.8 | 60.0 | -37% || 0.47 | 0.49 | +3% | 0.0926 | +| 33b || 69.4 | 56.8 | -18% || 0.47 | 0.49 | +3% | 0.3037 | +| 33c || 86.5 | 68.3 | -21% || 0.47 | 0.49 | +3% | 0.2026 | -| 3a || 344.9 | 398.9 | +16% || 0.47 | 0.49 | +3% | 0.2204 | -| 3b || 62.5 | 83.1 | +33% || 0.47 | 0.49 | +3% | 0.3535 | +| 3c || 551.8 | 505.8 | -8% || 0.47 | 0.49 | +3% | 0.1883 | +| 4a || 241.3 | 211.8 | -12% || 0.47 | 0.49 | +3% | 0.3983 | +| 4b || 49.8 | 45.7 | -8% || 0.47 | 0.49 | +3% | 0.7975 | | 4c || 223.3 | 214.9 | -4% || 0.47 | 0.49 | +3% | 0.7529 | +| 5a || 168.5 | 161.0 | -5% || 0.47 | 0.49 | +3% | 0.7354 | | 5b || 192.6 | 185.9 | -4% || 0.47 | 0.49 | +3% | 0.7479 | | 5c || 232.2 | 228.2 | -2% || 0.47 | 0.49 | +3% | 0.8850 | +| 6a || 122.7 | 106.1 | -14% || 0.47 | 0.49 | +3% | 0.4062 | +| 6b || 151.4 | 125.5 | -17% || 0.47 | 0.49 | +3% | 0.0824 | +| 6c || 122.6 | 101.8 | -17% || 0.47 | 0.49 | +3% | 0.3106 | +| 6d || 596.6 | 521.3 | -13% || 0.47 | 0.49 | +3% | 0.0499 | -| 6e || 122.1 | 130.0 | +6% || 0.47 | 0.49 | +3% | 0.7595 | | 6f || 989.2 | 946.2 | -4% || 0.47 | 0.49 | +3% | 0.2638 | +| 7a || 140.5 | 108.9 | -23% || 0.47 | 0.49 | +3% | 0.0834 | -| 7b || 100.1 | 137.2 | +37% || 0.47 | 0.49 | +3% | 0.0735 | +| 7c || 930.7 | 832.5 | -11% || 0.47 | 0.49 | +3% | 0.0418 | +| 8a || 157.8 | 150.0 | -5% || 0.47 | 0.49 | +3% | 0.7689 | +| 8b || 149.4 | 133.4 | -11% || 0.47 | 0.49 | +3% | 0.5097 | | 8c || 2364.4 | 2382.9 | +1% || 0.47 | 0.49 | +3% | 0.7739 | | 8d || 668.3 | 652.0 | -2% || 0.47 | 0.49 | +3% | 0.7382 | | 9a || 500.4 | 516.6 | +3% || 0.47 | 0.49 | +3% | 0.7117 | +| 9b || 378.0 | 360.8 | -5% || 0.47 | 0.49 | +3% | 0.6813 | -| 9c || 656.1 | 692.3 | +6% || 0.47 | 0.49 | +3% | 0.4686 | | 9d || 960.7 | 938.2 | -2% || 0.47 | 0.49 | +3% | 0.6911 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 54167.9 | 53273.9 | -2% || | | | | | Geomean || | | || | | +3% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

[mweisgut]

[5] Master vs. RobinMap (Iteration 2)

System Command:

../scripts/benchmark_all.sh 1c24c5f49719e589324037c5037866a7388a034c bbe2301e731486a993dcb9b2c15b62fbed65c5c5

nemea - click to expand

| property | value | | -- | -- | | Hostname | nemea | | CPU | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | Memory | 1.2TB | | numactl | nodebind: 1 | | numactl | membind: 1 |

Commit Info and Build Time	commit	date	message	build time
1c24c5f49	21.07.2022 21:52	Ensure correct results of EXTRACT expressions (#2482)	real 333.27 user 2773.77 sys 79.86
bbe2301e7	21.07.2022 13:31	Exchanged emplace with try_emplace	real 332.85 user 2770.95 sys 78.74

hyriseBenchmarkTPCH - single-threaded, SF 10.0

Sum of avg. item runtimes: +9% || Geometric mean of throughput changes: -7%

Configuration Overview - click to expand

```diff +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 06:17:41 | 2022-07-25 17:20:52 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 5719.0 | 5880.2 | +3% || 0.17 | 0.17 | -3% | 0.2090 | -| TPC-H 02 || 43.4 | 48.2 | +11%˄ || 23.04 | 20.73 | -10%˄ | 0.0000 | -| TPC-H 03 || 2227.8 | 2801.2 | +26% || 0.45 | 0.36 | -20% | 0.0000 | -| TPC-H 04 || 1397.4 | 1627.6 | +16% || 0.72 | 0.61 | -14% | 0.0000 | -| TPC-H 05 || 3123.6 | 3482.6 | +11% || 0.32 | 0.29 | -10% | 0.0000 | | TPC-H 06 || 153.6 | 152.6 | -1%˄ || 6.51 | 6.55 | +1%˄ | 0.3717 | -| TPC-H 07 || 928.2 | 1074.4 | +16% || 1.08 | 0.93 | -14% | 0.0000 | -| TPC-H 08 || 705.3 | 827.7 | +17% || 1.42 | 1.21 | -15% | 0.0000 | | TPC-H 09 || 5221.9 | 5350.4 | +2% || 0.19 | 0.19 | -2% | 0.0000 | -| TPC-H 10 || 2964.3 | 3251.2 | +10% || 0.34 | 0.31 | -9% | 0.0000 | -| TPC-H 11 || 73.4 | 78.8 | +7%˄ || 13.61 | 12.68 | -7%˄ | 0.0000 | -| TPC-H 12 || 989.9 | 1118.9 | +13% || 1.01 | 0.89 | -12% | 0.0000 | -| TPC-H 13 || 4693.6 | 4981.9 | +6% || 0.21 | 0.20 | -6% | 0.0000 | | TPC-H 14 || 420.9 | 429.8 | +2%˄ || 2.38 | 2.33 | -2%˄ | 0.0000 | | TPC-H 15 || 190.4 | 192.8 | +1%˄ || 5.25 | 5.19 | -1%˄ | 0.0000 | | TPC-H 16 || 631.8 | 624.4 | -1% || 1.58 | 1.60 | +1% | 0.0000 | | TPC-H 17 || 215.2 | 220.4 | +2%˄ || 4.65 | 4.54 | -2%˄ | 0.0000 | | TPC-H 18 || 1586.5 | 1538.6 | -3% || 0.63 | 0.65 | +3% | 0.0004 | | TPC-H 19 || 268.8 | 269.4 | +0%˄ || 3.72 | 3.71 | -0%˄ | 0.4273 | | TPC-H 20 || 400.2 | 407.9 | +2%˄ || 2.50 | 2.45 | -2%˄ | 0.0000 | -| TPC-H 21 || 4719.4 | 5438.9 | +15% || 0.21 | 0.18 | -13% | 0.0000 | -| TPC-H 22 || 426.1 | 477.0 | +12%˄ || 2.35 | 2.10 | -11%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 37100.7 | 40275.0 | +9% || | | | | -| Geomean || | | || | | -7% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - single-threaded, SF 0.01

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: -2%

Configuration Overview - click to expand

```diff +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st_s01.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st_s01.json | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 06:37:01 | 2022-07-25 17:40:36 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 0.009999999776482582 | 0.009999999776482582 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 4.8 | 4.9 | +2%˄ || 209.53 | 205.53 | -2%˄ | 0.0000 | | TPC-H 02 || 3.8 | 3.6 | -4%˄ || 265.42 | 277.34 | +4%˄ | 0.4809 | -| TPC-H 03 || 0.6 | 0.7 | +12%˄ || 1558.18 | 1389.96 | -11%˄ | 0.0000 | -| TPC-H 04 || 0.4 | 0.5 | +15%˄ || 2348.40 | 2051.03 | -13%˄ | 0.0000 | -| TPC-H 05 || 1.0 | 1.0 | +5%˄ || 1012.76 | 962.52 | -5%˄ | 0.0000 | | TPC-H 06 || 0.2 | 0.2 | +0%˄ || 5698.95 | 5670.42 | -1%˄ | 0.5191 | +| TPC-H 07 || 11.4 | 10.8 | -5%˄ || 87.64 | 92.52 | +6%˄ | 0.3731 | | TPC-H 08 || 15.4 | 15.8 | +3%˄ || 65.08 | 63.37 | -3%˄ | 0.1746 | +| TPC-H 09 || 3.8 | 3.3 | -12%˄ || 265.69 | 302.56 | +14%˄ | 0.3812 | -| TPC-H 10 || 0.8 | 0.8 | +5%˄ || 1286.16 | 1223.54 | -5%˄ | 0.0000 | | TPC-H 11 || 0.2 | 0.2 | +2%˄ || 4784.21 | 4697.71 | -2%˄ | 0.1432 | | TPC-H 12 || 0.6 | 0.6 | -1%˄ || 1607.29 | 1623.21 | +1%˄ | 0.0167 | | TPC-H 13 || 2.0 | 2.0 | -2%˄ || 500.28 | 508.22 | +2%˄ | 0.0000 | | TPC-H 14 || 0.3 | 0.3 | -0%˄ || 2944.02 | 2958.44 | +0%˄ | 0.6235 | | TPC-H 15 || 1.1 | 1.1 | +1%˄ || 893.19 | 884.59 | -1%˄ | 0.0000 | | TPC-H 16 || 1.9 | 1.9 | +0%˄ || 518.28 | 516.96 | -0%˄ | 0.3798 | -| TPC-H 17 || 0.6 | 0.7 | +7%˄ || 1536.66 | 1435.09 | -7%˄ | 0.5619 | | TPC-H 18 || 1.1 | 1.2 | +3%˄ || 883.86 | 857.00 | -3%˄ | 0.0000 | | TPC-H 19 || 4.8 | 4.8 | +1%˄ || 209.29 | 207.45 | -1%˄ | 0.0000 | -| TPC-H 20 || 2.0 | 2.1 | +6%˄ || 494.87 | 467.14 | -6%˄ | 0.0120 | | TPC-H 21 || 1.4 | 1.5 | +4%˄ || 699.80 | 674.25 | -4%˄ | 0.4214 | | TPC-H 22 || 1.0 | 1.0 | -1%˄ || 1010.76 | 1018.47 | +1%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 59.3 | 59.1 | -0% || | | | | | Geomean || | | || | | -2% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0

Sum of avg. item runtimes: +4% || Geometric mean of throughput changes: -3%

Configuration Overview - click to expand

```diff +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt_ordered.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt_ordered.json | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 06:37:29 | 2022-07-25 17:41:04 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | TPC-H 01 || 4748.2 | 4820.1 | +2% || 0.20 | 0.20 | +0% | (run time too short) | | TPC-H 02 || 63.0 | 64.6 | +3% || 14.55 | 14.21 | -2% | 0.0153 | -| TPC-H 03 || 1120.2 | 1193.2 | +7% || 0.88 | 0.83 | -6% | 0.0000 | -| TPC-H 04 || 595.7 | 676.2 | +14% || 1.65 | 1.47 | -11% | (run time too short) | | TPC-H 05 || 1022.1 | 1039.8 | +2% || 0.97 | 0.95 | -2% | 0.2818 | | TPC-H 06 || 72.1 | 71.9 | -0% || 12.95 | 12.95 | +0% | 0.4538 | | TPC-H 07 || 462.8 | 480.2 | +4% || 2.13 | 2.05 | -4% | 0.0000 | -| TPC-H 08 || 401.9 | 442.4 | +10% || 2.45 | 2.23 | -9% | 0.0000 | | TPC-H 09 || 2531.8 | 2536.9 | +0% || 0.38 | 0.38 | +0% | (run time too short) | -| TPC-H 10 || 1599.6 | 1673.7 | +5% || 0.62 | 0.58 | -5% | (run time too short) | | TPC-H 11 || 90.2 | 92.4 | +2% || 10.50 | 10.25 | -2% | 0.0001 | -| TPC-H 12 || 550.9 | 593.9 | +8% || 1.78 | 1.67 | -7% | 0.0000 | | TPC-H 13 || 3005.3 | 3092.4 | +3% || 0.32 | 0.32 | +0% | (run time too short) | | TPC-H 14 || 171.6 | 173.9 | +1% || 5.65 | 5.58 | -1% | 0.0092 | | TPC-H 15 || 169.8 | 168.3 | -1% || 5.70 | 5.77 | +1% | 0.0955 | | TPC-H 16 || 775.2 | 766.2 | -1% || 1.27 | 1.28 | +1% | (run time too short) | | TPC-H 17 || 93.6 | 95.2 | +2% || 10.15 | 9.97 | -2% | 0.0004 | | TPC-H 18 || 2455.3 | 2554.4 | +4% || 0.40 | 0.38 | -4% | (run time too short) | | TPC-H 19 || 156.7 | 157.1 | +0% || 6.17 | 6.15 | -0% | 0.6192 | | TPC-H 20 || 254.7 | 251.3 | -1% || 3.83 | 3.90 | +2% | 0.0526 | -| TPC-H 21 || 1201.4 | 1348.8 | +12% || 0.82 | 0.73 | -10% | (run time too short) | -| TPC-H 22 || 153.6 | 174.1 | +13% || 6.28 | 5.58 | -11% | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 21695.7 | 22467.1 | +4% || | | | | | Geomean || | | || | | -3% | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0

Sum of avg. item runtimes: +9% || Geometric mean of throughput changes: -10%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 07:01:13 | 2022-07-25 18:04:55 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ -| TPC-H 01 || 5506.5 | 5736.3 | +4% || 0.47 | 0.42 | -10% | 0.0051 | -| TPC-H 02 || 343.7 | 387.4 | +13% || 0.47 | 0.42 | -10% | 0.5197 | -| TPC-H 03 || 3434.3 | 4211.7 | +23% || 0.47 | 0.42 | -10% | 0.0002 | -| TPC-H 04 || 2574.0 | 2745.7 | +7% || 0.47 | 0.42 | -10% | 0.4030 | -| TPC-H 05 || 4014.3 | 4752.9 | +18% || 0.47 | 0.42 | -10% | 0.0008 | -| TPC-H 06 || 462.6 | 462.7 | +0% || 0.47 | 0.42 | -10% | 0.9977 | -| TPC-H 07 || 2434.9 | 2785.8 | +14% || 0.47 | 0.42 | -10% | 0.0488 | -| TPC-H 08 || 1866.3 | 2087.5 | +12% || 0.47 | 0.42 | -10% | 0.1565 | -| TPC-H 09 || 5874.3 | 6102.2 | +4% || 0.47 | 0.42 | -9% | 0.2938 | -| TPC-H 10 || 4686.0 | 5036.6 | +7% || 0.47 | 0.42 | -10% | 0.0852 | -| TPC-H 11 || 546.9 | 562.2 | +3% || 0.47 | 0.42 | -10% | 0.8611 | -| TPC-H 12 || 2113.7 | 2315.2 | +10% || 0.47 | 0.42 | -10% | 0.1228 | -| TPC-H 13 || 5660.5 | 6174.9 | +9% || 0.47 | 0.42 | -10% | 0.0001 | -| TPC-H 14 || 1054.6 | 1136.4 | +8% || 0.47 | 0.42 | -10% | 0.4740 | +| TPC-H 15 || 548.8 | 497.4 | -9% || 0.47 | 0.42 | -10% | 0.4498 | -| TPC-H 16 || 2267.7 | 2324.3 | +2% || 0.47 | 0.42 | -10% | 0.7205 | -| TPC-H 17 || 494.6 | 568.8 | +15% || 0.47 | 0.42 | -10% | 0.2665 | -| TPC-H 18 || 3510.3 | 3649.4 | +4% || 0.47 | 0.42 | -10% | 0.1658 | -| TPC-H 19 || 722.9 | 794.0 | +10% || 0.47 | 0.42 | -10% | 0.4558 | -| TPC-H 20 || 1168.6 | 1159.2 | -1% || 0.47 | 0.42 | -10% | 0.9158 | -| TPC-H 21 || 5697.1 | 6200.2 | +9% || 0.47 | 0.42 | -10% | 0.0529 | -| TPC-H 22 || 748.1 | 1043.5 | +39% || 0.47 | 0.42 | -10% | 0.0067 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 55730.7 | 60734.4 | +9% || | | | | -| Geomean || | | || | | -10% | | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - single-threaded

Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -9%

Configuration Overview - click to expand

```diff +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 07:22:56 | 2022-07-25 18:26:41 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 258.4 | 262.6 | +2%˄ || 3.87 | 3.81 | -2%˄ | 0.0000 | -| 03 || 69.1 | 79.6 | +15%˄ || 14.47 | 12.57 | -13%˄ | 0.0000 | -| 06 || 151.1 | 167.8 | +11%˄ || 6.62 | 5.96 | -10%˄ | 0.0000 | -| 07 || 297.7 | 316.1 | +6%˄ || 3.36 | 3.16 | -6%˄ | 0.0000 | | 09 || 631.7 | 630.1 | -0% || 1.58 | 1.59 | +0% | 0.0000 | -| 10 || 131.9 | 161.2 | +22%˄ || 7.58 | 6.20 | -18%˄ | 0.0000 | | 13 || 422.9 | 439.5 | +4%˄ || 2.36 | 2.28 | -4%˄ | 0.0000 | -| 15 || 102.6 | 113.5 | +11%˄ || 9.75 | 8.81 | -10%˄ | 0.0000 | | 16 || 152.9 | 158.5 | +4%˄ || 6.54 | 6.31 | -4%˄ | 0.0000 | -| 17 || 306.2 | 337.4 | +10%˄ || 3.27 | 2.96 | -9%˄ | 0.0000 | -| 19 || 99.8 | 119.7 | +20%˄ || 10.02 | 8.36 | -17%˄ | 0.0000 | -| 25 || 163.2 | 192.5 | +18%˄ || 6.13 | 5.19 | -15%˄ | 0.0000 | | 26 || 130.1 | 134.0 | +3%˄ || 7.69 | 7.46 | -3%˄ | 0.0000 | | 28 || 603.1 | 602.9 | -0%˄ || 1.66 | 1.66 | +0%˄ | 0.8495 | -| 29 || 468.6 | 497.9 | +6%˄ || 2.13 | 2.01 | -6%˄ | 0.0000 | | 31 || 1262.8 | 1317.3 | +4% || 0.79 | 0.76 | -4% | 0.0000 | -| 32 || 40.7 | 50.4 | +24%˄ || 24.56 | 19.85 | -19%˄ | 0.0000 | -| 34 || 157.6 | 185.0 | +17%˄ || 6.35 | 5.41 | -15%˄ | 0.0000 | -| 35 || 605.7 | 673.6 | +11%˄ || 1.65 | 1.48 | -10%˄ | 0.0000 | -| 37 || 274.9 | 326.0 | +19%˄ || 3.64 | 3.07 | -16%˄ | 0.0000 | | 39a || 1751.2 | 1774.5 | +1% || 0.57 | 0.56 | -1% | 0.0033 | | 39b || 1701.8 | 1757.9 | +3% || 0.59 | 0.57 | -3% | 0.0000 | | 41 || 288.8 | 287.0 | -1%˄ || 3.46 | 3.48 | +1%˄ | 0.0000 | -| 42 || 81.4 | 100.4 | +23%˄ || 12.28 | 9.96 | -19%˄ | 0.0000 | | 43 || 961.6 | 951.2 | -1% || 1.04 | 1.05 | +1% | 0.0045 | -| 45 || 113.5 | 121.1 | +7%˄ || 8.81 | 8.26 | -6%˄ | 0.0000 | | 48 || 1011.2 | 1033.3 | +2% || 0.99 | 0.97 | -2% | 0.0000 | -| 50 || 108.1 | 129.1 | +19%˄ || 9.25 | 7.75 | -16%˄ | 0.0000 | -| 52 || 78.3 | 96.5 | +23%˄ || 12.77 | 10.36 | -19%˄ | 0.0000 | -| 55 || 75.0 | 93.7 | +25%˄ || 13.32 | 10.67 | -20%˄ | 0.0000 | | 62 || 542.2 | 530.1 | -2%˄ || 1.84 | 1.89 | +2%˄ | 0.0000 | | 65 || 1757.9 | 1748.0 | -1% || 0.57 | 0.57 | +1% | 0.2988 | -| 69 || 119.2 | 151.2 | +27%˄ || 8.39 | 6.62 | -21%˄ | 0.0000 | -| 73 || 76.6 | 94.8 | +24%˄ || 13.05 | 10.55 | -19%˄ | 0.0000 | | 79 || 467.0 | 483.7 | +4%˄ || 2.14 | 2.07 | -3%˄ | 0.0000 | | 81 || 206.5 | 206.7 | +0%˄ || 4.84 | 4.84 | -0%˄ | 0.6640 | -| 82 || 355.1 | 425.0 | +20%˄ || 2.82 | 2.35 | -16%˄ | 0.0000 | -| 83 || 39.9 | 42.9 | +7%˄ || 25.05 | 23.31 | -7%˄ | 0.0000 | | 85 || 133.9 | 132.7 | -1%˄ || 7.47 | 7.54 | +1%˄ | 0.0000 | -| 88 || 615.5 | 799.9 | +30% || 1.62 | 1.25 | -23% | 0.0000 | -| 91 || 15.3 | 17.6 | +15%˄ || 65.16 | 56.72 | -13%˄ | 0.0000 | -| 92 || 34.5 | 40.0 | +16%˄ || 28.99 | 24.98 | -14%˄ | 0.0000 | | 93 || 3862.6 | 3826.0 | -1% || 0.26 | 0.26 | +1% | 0.0085 | -| 94 || 97.7 | 104.4 | +7%˄ || 10.23 | 9.58 | -6%˄ | 0.0000 | | 95 || 9072.9 | 8998.6 | -1% || 0.11 | 0.11 | +1% | ˅ | -| 96 || 56.9 | 76.8 | +35%˄ || 17.56 | 13.02 | -26%˄ | 0.0000 | | 97 || 3255.5 | 3242.8 | -0% || 0.31 | 0.31 | +0% | 0.6070 | | 99 || 1043.2 | 1023.1 | -2% || 0.96 | 0.98 | +2% | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 34254.4 | 35054.5 | +2% || | | | | -| Geomean || | | || | | -9% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | | || ˅ Insufficient number of runs for p-value calculation | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -2%

Configuration Overview - click to expand

```diff +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 07:48:56 | 2022-07-25 18:53:37 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 632.8 | 651.2 | +3% || 0.47 | 0.46 | -2% | 0.7697 | -| 03 || 157.3 | 198.4 | +26% || 0.47 | 0.46 | -2% | 0.2162 | | 06 || 707.8 | 738.4 | +4% || 0.47 | 0.46 | -2% | 0.7511 | | 07 || 862.1 | 835.9 | -3% || 0.47 | 0.46 | -2% | 0.7752 | +| 09 || 1292.9 | 1218.8 | -6% || 0.47 | 0.46 | -2% | 0.3020 | -| 10 || 412.1 | 571.2 | +39% || 0.47 | 0.46 | -2% | 0.0752 | | 13 || 1615.1 | 1577.9 | -2% || 0.47 | 0.46 | -2% | 0.7999 | -| 15 || 404.2 | 451.6 | +12% || 0.47 | 0.46 | -2% | 0.4764 | +| 16 || 826.2 | 632.3 | -23% || 0.47 | 0.46 | -2% | 0.0880 | -| 17 || 884.6 | 1001.3 | +13% || 0.47 | 0.46 | -2% | 0.2362 | +| 19 || 532.5 | 420.9 | -21% || 0.47 | 0.46 | -2% | 0.2086 | -| 25 || 722.1 | 794.8 | +10% || 0.47 | 0.46 | -2% | 0.4613 | | 26 || 491.2 | 484.1 | -1% || 0.47 | 0.46 | -2% | 0.9033 | -| 28 || 530.2 | 612.5 | +16% || 0.47 | 0.46 | -2% | 0.2590 | -| 29 || 1275.6 | 1420.5 | +11% || 0.47 | 0.46 | -2% | 0.2017 | +| 31 || 1919.4 | 1803.0 | -6% || 0.47 | 0.46 | -2% | 0.4210 | -| 32 || 333.7 | 355.7 | +7% || 0.47 | 0.46 | -2% | 0.7673 | -| 34 || 461.1 | 508.0 | +10% || 0.47 | 0.46 | -2% | 0.4947 | -| 35 || 1781.8 | 1953.3 | +10% || 0.47 | 0.46 | -2% | 0.2576 | -| 37 || 501.8 | 575.8 | +15% || 0.47 | 0.46 | -2% | 0.2532 | | 39a || 2086.2 | 2089.5 | +0% || 0.47 | 0.46 | -2% | 0.9766 | | 39b || 2145.1 | 2098.3 | -2% || 0.47 | 0.46 | -2% | 0.6680 | +| 41 || 2973.2 | 2827.2 | -5% || 0.47 | 0.46 | -2% | 0.4171 | -| 42 || 269.8 | 368.9 | +37% || 0.47 | 0.46 | -2% | 0.0857 | -| 43 || 1284.9 | 1454.7 | +13% || 0.47 | 0.46 | -2% | 0.0836 | | 45 || 527.7 | 539.0 | +2% || 0.47 | 0.46 | -2% | 0.8989 | -| 48 || 2544.0 | 2715.4 | +7% || 0.47 | 0.46 | -2% | 0.2940 | -| 50 || 504.0 | 583.9 | +16% || 0.47 | 0.46 | -2% | 0.2564 | | 52 || 342.2 | 331.4 | -3% || 0.47 | 0.46 | -2% | 0.8429 | -| 55 || 247.1 | 305.8 | +24% || 0.47 | 0.46 | -2% | 0.1834 | | 62 || 839.7 | 847.9 | +1% || 0.47 | 0.46 | -2% | 0.9114 | | 65 || 3586.8 | 3701.4 | +3% || 0.47 | 0.46 | -2% | 0.2449 | -| 69 || 514.2 | 716.8 | +39% || 0.47 | 0.46 | -2% | 0.0581 | +| 73 || 298.3 | 269.4 | -10% || 0.47 | 0.46 | -2% | 0.5400 | -| 79 || 1114.7 | 1205.0 | +8% || 0.47 | 0.46 | -2% | 0.2993 | | 81 || 787.6 | 797.3 | +1% || 0.47 | 0.46 | -2% | 0.9173 | | 82 || 752.0 | 773.3 | +3% || 0.47 | 0.46 | -2% | 0.7815 | +| 83 || 255.2 | 240.7 | -6% || 0.47 | 0.46 | -2% | 0.7546 | +| 85 || 685.4 | 634.9 | -7% || 0.47 | 0.46 | -2% | 0.5154 | | 88 || 752.5 | 744.0 | -1% || 0.47 | 0.46 | -2% | 0.9190 | +| 91 || 162.9 | 146.8 | -10% || 0.47 | 0.46 | -2% | 0.7167 | | 92 || 257.5 | 261.6 | +2% || 0.47 | 0.46 | -2% | 0.9361 | | 93 || 2417.5 | 2324.3 | -4% || 0.47 | 0.46 | -2% | 0.4606 | +| 94 || 790.1 | 733.0 | -7% || 0.47 | 0.46 | -2% | 0.6174 | | 95 || 8050.9 | 8165.8 | +1% || 0.47 | 0.46 | -2% | 0.5423 | +| 96 || 209.0 | 197.6 | -5% || 0.47 | 0.46 | -2% | 0.7440 | | 97 || 4343.2 | 4311.2 | -1% || 0.47 | 0.46 | -2% | 0.8372 | | 99 || 1386.1 | 1362.8 | -2% || 0.47 | 0.46 | -2% | 0.8052 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 56472.4 | 57553.4 | +2% || | | | | | Geomean || | | || | | -2% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - single-threaded

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: +0%

Configuration Overview - click to expand

```diff +Configuration Overview---+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 08:09:25 | 2022-07-25 19:14:04 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 22.8 | 22.9 | +0% || 4.25 | 4.27 | +0% | 0.6942 | | New-Order || 16.4 | 16.4 | -0% || 47.94 | 47.94 | +0% | 0.8656 | | Order-Status || 1.1 | 1.1 | +0% || 4.27 | 4.27 | +0% | 0.6240 | | Payment || 2.1 | 2.1 | +0% || 45.82 | 45.81 | -0% | 0.7432 | | Stock-Level || 3.5 | 3.6 | +3% || 4.27 | 4.27 | +0% | 0.0049 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 45.9 | 46.0 | +0% || | | | | | Geomean || | | || | | +0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: +0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 08:10:26 | 2022-07-25 19:15:05 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 218.7 | 219.2 | +0% || 4.34 | 4.32 | -0% | 0.7313 | | unsucc.: || 4.4 | 4.3 | -0% || 62.89 | 62.89 | -0% | | | New-Order || 57.2 | 57.2 | +0% || 133.48 | 134.42 | +1% | 0.5019 | | unsucc.: || 5.0 | 4.9 | -2% || 622.86 | 621.63 | -0% | | | Order-Status || 7.5 | 7.5 | -1% || 67.23 | 67.20 | -0% | 0.1201 | | Payment || 9.9 | 9.9 | +0% || 21.43 | 21.40 | -0% | 0.8155 | | unsucc.: || 4.4 | 4.4 | -1% || 701.30 | 701.05 | -0% | | | Stock-Level || 17.3 | 17.6 | +2% || 67.23 | 67.20 | -0% | 0.0001 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 310.5 | 311.4 | +0% || | | | | | Geomean || | | || | | +0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - single-threaded

Sum of avg. item runtimes: +4% || Geometric mean of throughput changes: -11%

Configuration Overview - click to expand

```diff +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_st.json | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 08:30:34 | 2022-07-25 19:35:12 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+----------------------+ -| 10a || 168.1 | 177.8 | +6%˄ || 5.95 | 5.62 | -5%˄ | 0.0000 | -| 10b || 116.1 | 128.0 | +10%˄ || 8.61 | 7.81 | -9%˄ | 0.0000 | | 10c || 323.1 | 322.5 | -0%˄ || 3.09 | 3.10 | +0%˄ | 0.7127 | -| 11a || 15.2 | 20.1 | +32%˄ || 65.90 | 49.83 | -24%˄ | 0.0000 | -| 11b || 15.0 | 20.0 | +33%˄ || 66.65 | 50.00 | -25%˄ | 0.0000 | -| 11c || 20.6 | 25.3 | +23%˄ || 48.47 | 39.57 | -18%˄ | 0.0000 | -| 11d || 23.6 | 28.0 | +18%˄ || 42.33 | 35.73 | -16%˄ | 0.0000 | | 12a || 48.7 | 50.2 | +3%˄ || 20.52 | 19.91 | -3%˄ | 0.0000 | -| 12b || 29.3 | 40.5 | +38%˄ || 34.16 | 24.70 | -28%˄ | 0.0000 | -| 12c || 83.4 | 87.5 | +5%˄ || 11.99 | 11.43 | -5%˄ | 0.0000 | | 13a || 277.0 | 286.8 | +4%˄ || 3.61 | 3.49 | -3%˄ | 0.0000 | -| 13b || 154.2 | 170.0 | +10%˄ || 6.48 | 5.88 | -9%˄ | 0.0000 | -| 13c || 115.7 | 132.6 | +15%˄ || 8.64 | 7.54 | -13%˄ | 0.0000 | | 13d || 589.4 | 576.3 | -2%˄ || 1.70 | 1.74 | +2%˄ | 0.0000 | | 14a || 415.8 | 424.4 | +2%˄ || 2.40 | 2.36 | -2%˄ | 0.0000 | -| 14b || 233.6 | 252.3 | +8%˄ || 4.28 | 3.96 | -7%˄ | 0.0000 | | 14c || 491.2 | 500.3 | +2%˄ || 2.04 | 2.00 | -2%˄ | 0.0000 | -| 15a || 81.6 | 89.3 | +9%˄ || 12.26 | 11.20 | -9%˄ | 0.0000 | -| 15b || 80.8 | 88.5 | +10%˄ || 12.38 | 11.30 | -9%˄ | 0.0000 | -| 15c || 85.8 | 93.6 | +9%˄ || 11.65 | 10.69 | -8%˄ | 0.0000 | -| 15d || 83.7 | 90.8 | +9%˄ || 11.95 | 11.01 | -8%˄ | 0.0000 | | 16a || 2259.8 | 2308.1 | +2% || 0.44 | 0.43 | -2% | 0.0000 | | 16b || 3294.0 | 3353.1 | +2% || 0.30 | 0.30 | -2% | 0.0004 | | 16c || 2359.9 | 2404.0 | +2% || 0.42 | 0.42 | -2% | 0.0000 | | 16d || 2344.3 | 2380.2 | +2% || 0.43 | 0.42 | -2% | 0.0001 | -| 17a || 585.4 | 616.5 | +5%˄ || 1.71 | 1.62 | -5%˄ | (run time too short) | -| 17b || 460.8 | 495.9 | +8%˄ || 2.17 | 2.02 | -7%˄ | 0.0000 | -| 17c || 441.0 | 473.6 | +7%˄ || 2.27 | 2.11 | -7%˄ | 0.0000 | -| 17d || 522.1 | 554.1 | +6%˄ || 1.92 | 1.80 | -6%˄ | 0.0000 | | 17e || 1611.1 | 1616.6 | +0% || 0.62 | 0.62 | -0% | 0.0763 | | 17f || 947.3 | 980.9 | +4% || 1.06 | 1.02 | -3% | 0.0000 | -| 18a || 599.8 | 641.3 | +7%˄ || 1.67 | 1.56 | -6%˄ | 0.0000 | -| 18b || 112.7 | 123.0 | +9%˄ || 8.87 | 8.13 | -8%˄ | 0.0000 | | 18c || 564.2 | 580.8 | +3%˄ || 1.77 | 1.72 | -3%˄ | 0.0000 | -| 19a || 180.6 | 193.3 | +7%˄ || 5.54 | 5.17 | -7%˄ | 0.0000 | -| 19b || 124.8 | 136.7 | +10%˄ || 8.01 | 7.32 | -9%˄ | 0.0000 | -| 19c || 193.9 | 205.6 | +6%˄ || 5.16 | 4.86 | -6%˄ | 0.0000 | | 19d || 720.8 | 736.5 | +2% || 1.39 | 1.36 | -2% | 0.0000 | -| 1a || 9.0 | 12.0 | +33%˄ || 111.54 | 83.67 | -25%˄ | 0.0000 | -| 1b || 8.4 | 11.4 | +36%˄ || 119.22 | 87.72 | -26%˄ | 0.0000 | -| 1c || 9.5 | 12.5 | +32%˄ || 105.47 | 80.14 | -24%˄ | 0.0000 | -| 1d || 8.4 | 11.4 | +36%˄ || 119.33 | 87.44 | -27%˄ | 0.0000 | | 20a || 393.5 | 401.4 | +2%˄ || 2.54 | 2.49 | -2%˄ | 0.0000 | -| 20b || 229.4 | 242.6 | +6%˄ || 4.36 | 4.12 | -5%˄ | 0.0000 | -| 20c || 221.7 | 234.2 | +6%˄ || 4.51 | 4.27 | -5%˄ | 0.0000 | -| 21a || 35.2 | 48.4 | +37%˄ || 28.37 | 20.66 | -27%˄ | 0.0000 | -| 21b || 19.1 | 24.2 | +27%˄ || 52.29 | 41.33 | -21%˄ | 0.0000 | -| 21c || 34.7 | 48.0 | +38%˄ || 28.79 | 20.84 | -28%˄ | 0.0000 | -| 22a || 196.4 | 215.1 | +10%˄ || 5.09 | 4.65 | -9%˄ | 0.0000 | -| 22b || 143.2 | 156.3 | +9%˄ || 6.98 | 6.40 | -8%˄ | 0.0000 | -| 22c || 492.3 | 514.6 | +5%˄ || 2.03 | 1.94 | -4%˄ | 0.0000 | | 22d || 942.4 | 927.6 | -2% || 1.06 | 1.08 | +2% | 0.0000 | -| 23a || 42.8 | 54.7 | +28%˄ || 23.34 | 18.29 | -22%˄ | 0.0000 | -| 23b || 44.1 | 55.3 | +25%˄ || 22.66 | 18.07 | -20%˄ | 0.0000 | -| 23c || 85.8 | 93.7 | +9%˄ || 11.66 | 10.68 | -8%˄ | 0.0000 | -| 24a || 145.1 | 167.9 | +16%˄ || 6.89 | 5.96 | -14%˄ | 0.0000 | -| 24b || 123.4 | 147.2 | +19%˄ || 8.11 | 6.79 | -16%˄ | 0.0000 | -| 25a || 278.1 | 291.6 | +5%˄ || 3.60 | 3.43 | -5%˄ | 0.0000 | -| 25b || 113.6 | 135.9 | +20%˄ || 8.81 | 7.36 | -16%˄ | 0.0000 | | 25c || 920.8 | 932.4 | +1% || 1.09 | 1.07 | -1% | 0.0000 | -| 26a || 164.1 | 178.3 | +9%˄ || 6.09 | 5.61 | -8%˄ | 0.0000 | -| 26b || 131.4 | 146.9 | +12%˄ || 7.61 | 6.81 | -11%˄ | 0.0000 | -| 26c || 233.9 | 247.4 | +6%˄ || 4.27 | 4.04 | -5%˄ | 0.0000 | -| 27a || 22.7 | 27.8 | +22%˄ || 43.99 | 36.00 | -18%˄ | 0.0000 | -| 27b || 22.4 | 27.4 | +23%˄ || 44.73 | 36.45 | -19%˄ | 0.0000 | -| 27c || 35.0 | 48.2 | +38%˄ || 28.55 | 20.76 | -27%˄ | 0.0000 | -| 28a || 291.2 | 304.6 | +5%˄ || 3.43 | 3.28 | -4%˄ | 0.0000 | -| 28b || 37.2 | 55.3 | +48%˄ || 26.85 | 18.09 | -33%˄ | 0.0000 | -| 28c || 329.3 | 347.4 | +6%˄ || 3.04 | 2.88 | -5%˄ | 0.0000 | -| 29a || 101.8 | 133.3 | +31%˄ || 9.82 | 7.50 | -24%˄ | 0.0000 | -| 29b || 76.4 | 106.4 | +39%˄ || 13.09 | 9.40 | -28%˄ | 0.0000 | -| 29c || 149.5 | 174.8 | +17%˄ || 6.69 | 5.72 | -14%˄ | 0.0000 | -| 2a || 46.4 | 50.9 | +10%˄ || 21.56 | 19.63 | -9%˄ | 0.0000 | -| 2b || 37.9 | 42.6 | +12%˄ || 26.41 | 23.48 | -11%˄ | 0.0000 | -| 2c || 21.9 | 26.9 | +23%˄ || 45.69 | 37.19 | -19%˄ | 0.0000 | -| 2d || 65.3 | 69.8 | +7%˄ || 15.32 | 14.32 | -7%˄ | 0.0000 | -| 30a || 150.1 | 163.9 | +9%˄ || 6.66 | 6.10 | -8%˄ | 0.0000 | -| 30b || 128.1 | 142.9 | +12%˄ || 7.81 | 7.00 | -10%˄ | 0.0000 | | 30c || 382.7 | 398.9 | +4%˄ || 2.61 | 2.51 | -4%˄ | 0.0000 | -| 31a || 146.5 | 171.6 | +17%˄ || 6.82 | 5.83 | -15%˄ | 0.0000 | -| 31b || 125.0 | 150.1 | +20%˄ || 8.00 | 6.66 | -17%˄ | 0.0000 | -| 31c || 170.4 | 195.5 | +15%˄ || 5.87 | 5.11 | -13%˄ | 0.0000 | -| 32a || 12.9 | 16.9 | +31%˄ || 77.48 | 59.21 | -24%˄ | 0.0000 | -| 32b || 30.9 | 37.0 | +20%˄ || 32.34 | 27.04 | -16%˄ | 0.0000 | -| 33a || 19.3 | 26.1 | +35%˄ || 51.70 | 38.35 | -26%˄ | 0.0000 | -| 33b || 18.7 | 25.5 | +36%˄ || 53.43 | 39.18 | -27%˄ | 0.0000 | -| 33c || 22.6 | 29.3 | +30%˄ || 44.30 | 34.11 | -23%˄ | 0.0000 | | 3a || 123.0 | 126.9 | +3%˄ || 8.13 | 7.88 | -3%˄ | 0.0000 | -| 3b || 12.7 | 16.3 | +28%˄ || 78.67 | 61.39 | -22%˄ | 0.0000 | -| 3c || 438.3 | 461.6 | +5%˄ || 2.28 | 2.17 | -5%˄ | 0.0000 | +| 4a || 113.5 | 107.2 | -6%˄ || 8.81 | 9.33 | +6%˄ | 0.0000 | -| 4b || 11.5 | 15.2 | +33%˄ || 87.12 | 65.61 | -25%˄ | 0.0000 | | 4c || 124.3 | 122.9 | -1%˄ || 8.04 | 8.13 | +1%˄ | 0.0000 | | 5a || 52.8 | 53.4 | +1%˄ || 18.93 | 18.74 | -1%˄ | 0.0000 | +| 5b || 135.0 | 125.3 | -7%˄ || 7.41 | 7.98 | +8%˄ | 0.0000 | -| 5c || 183.2 | 193.7 | +6%˄ || 5.46 | 5.16 | -5%˄ | 0.0000 | -| 6a || 103.9 | 117.9 | +14%˄ || 9.63 | 8.48 | -12%˄ | 0.0000 | -| 6b || 115.9 | 129.2 | +11%˄ || 8.63 | 7.74 | -10%˄ | 0.0000 | -| 6c || 98.7 | 112.9 | +14%˄ || 10.13 | 8.86 | -13%˄ | 0.0000 | | 6d || 531.1 | 550.5 | +4%˄ || 1.88 | 1.82 | -4%˄ | 0.0000 | -| 6e || 103.6 | 117.9 | +14%˄ || 9.65 | 8.48 | -12%˄ | 0.0000 | | 6f || 703.0 | 714.8 | +2% || 1.42 | 1.40 | -2% | 0.0000 | -| 7a || 53.3 | 77.8 | +46%˄ || 18.77 | 12.86 | -31%˄ | 0.0000 | -| 7b || 49.7 | 74.5 | +50%˄ || 20.13 | 13.42 | -33%˄ | 0.0000 | | 7c || 591.2 | 593.8 | +0%˄ || 1.69 | 1.68 | -0%˄ | 0.0618 | -| 8a || 38.2 | 43.0 | +12%˄ || 26.16 | 23.28 | -11%˄ | 0.0000 | -| 8b || 35.8 | 40.7 | +14%˄ || 27.95 | 24.56 | -12%˄ | 0.0000 | | 8c || 1873.7 | 1928.0 | +3% || 0.53 | 0.52 | -3% | 0.0000 | -| 8d || 306.2 | 346.8 | +13%˄ || 3.27 | 2.88 | -12%˄ | 0.0000 | | 9a || 187.5 | 191.5 | +2%˄ || 5.33 | 5.22 | -2%˄ | 0.0000 | | 9b || 105.1 | 108.7 | +3%˄ || 9.52 | 9.20 | -3%˄ | 0.0000 | | 9c || 270.1 | 273.8 | +1%˄ || 3.70 | 3.65 | -1%˄ | 0.0000 | | 9d || 476.1 | 479.0 | +1%˄ || 2.10 | 2.09 | -1%˄ | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 35082.5 | 36612.0 | +4% || | | | | -| Geomean || | | || | | -11% | | +---------++----------+---------+--------++----------+----------+--------+----------------------+ | Notes || ˄ Execution stopped due to max runs reached | +---------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +7% || Geometric mean of throughput changes: -8%

Configuration Overview - click to expand

```diff +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_bbe2301e731486a993dcb9b2c15b62fbed65c5c5_mt.json | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | bbe2301e731486a993dcb9b2c15b62fbed65c5c5-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 09:12:50 | 2022-07-25 20:19:24 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ -| 10a || 198.5 | 192.3 | -3% || 0.51 | 0.47 | -8% | 0.8041 | -| 10b || 123.3 | 163.1 | +32% || 0.51 | 0.47 | -8% | 0.0289 | -| 10c || 401.6 | 427.2 | +6% || 0.51 | 0.47 | -8% | 0.3781 | -| 11a || 49.0 | 54.7 | +12% || 0.51 | 0.47 | -8% | 0.5900 | -| 11b || 38.1 | 40.0 | +5% || 0.51 | 0.47 | -8% | 0.6964 | +| 11c || 101.2 | 60.3 | -40% || 0.51 | 0.47 | -8% | 0.0352 | -| 11d || 72.2 | 106.2 | +47% || 0.51 | 0.47 | -8% | 0.0896 | -| 12a || 146.0 | 193.4 | +32% || 0.51 | 0.47 | -8% | 0.0328 | +| 12b || 80.2 | 71.3 | -11% || 0.51 | 0.47 | -8% | 0.6465 | -| 12c || 313.5 | 325.7 | +4% || 0.51 | 0.47 | -8% | 0.7141 | -| 13a || 590.9 | 626.8 | +6% || 0.51 | 0.47 | -8% | 0.3337 | -| 13b || 263.5 | 319.4 | +21% || 0.51 | 0.47 | -8% | 0.0969 | -| 13c || 238.4 | 281.0 | +18% || 0.51 | 0.47 | -8% | 0.1961 | -| 13d || 971.2 | 1018.1 | +5% || 0.51 | 0.47 | -8% | 0.3409 | -| 14a || 763.4 | 837.9 | +10% || 0.51 | 0.47 | -8% | 0.0773 | -| 14b || 432.1 | 510.1 | +18% || 0.51 | 0.47 | -8% | 0.0269 | -| 14c || 928.0 | 1008.6 | +9% || 0.51 | 0.47 | -7% | 0.1453 | -| 15a || 164.8 | 173.4 | +5% || 0.51 | 0.47 | -8% | 0.7336 | +| 15b || 163.2 | 134.0 | -18% || 0.51 | 0.47 | -8% | 0.2326 | -| 15c || 155.3 | 155.9 | +0% || 0.51 | 0.47 | -8% | 0.9780 | -| 15d || 182.0 | 237.7 | +31% || 0.51 | 0.47 | -8% | 0.0645 | -| 16a || 2540.7 | 2707.6 | +7% || 0.51 | 0.47 | -8% | 0.0143 | -| 16b || 3719.8 | 3931.4 | +6% || 0.51 | 0.47 | -8% | 0.0001 | -| 16c || 2679.6 | 2776.0 | +4% || 0.51 | 0.47 | -8% | 0.1120 | -| 16d || 2581.4 | 2771.0 | +7% || 0.51 | 0.47 | -8% | 0.0012 | -| 17a || 785.0 | 850.9 | +8% || 0.51 | 0.47 | -8% | 0.1830 | +| 17b || 626.7 | 581.4 | -7% || 0.51 | 0.47 | -8% | 0.2990 | +| 17c || 546.4 | 479.4 | -12% || 0.51 | 0.47 | -8% | 0.1166 | -| 17d || 620.7 | 636.7 | +3% || 0.51 | 0.47 | -8% | 0.7301 | -| 17e || 1437.5 | 1598.9 | +11% || 0.51 | 0.47 | -8% | 0.0003 | -| 17f || 1012.8 | 1101.6 | +9% || 0.51 | 0.47 | -7% | 0.0813 | -| 18a || 712.1 | 785.8 | +10% || 0.51 | 0.47 | -8% | 0.1094 | +| 18b || 150.3 | 141.9 | -6% || 0.51 | 0.47 | -8% | 0.6010 | -| 18c || 607.0 | 728.2 | +20% || 0.51 | 0.47 | -8% | 0.0058 | -| 19a || 400.0 | 483.2 | +21% || 0.51 | 0.47 | -8% | 0.0487 | -| 19b || 326.4 | 357.1 | +9% || 0.51 | 0.47 | -8% | 0.3639 | -| 19c || 450.2 | 556.2 | +24% || 0.51 | 0.47 | -8% | 0.0203 | -| 19d || 1254.9 | 1315.4 | +5% || 0.51 | 0.47 | -8% | 0.3294 | -| 1a || 31.1 | 47.4 | +52% || 0.51 | 0.47 | -8% | 0.1731 | -| 1b || 29.5 | 34.3 | +16% || 0.51 | 0.47 | -8% | 0.5668 | +| 1c || 35.7 | 33.2 | -7% || 0.51 | 0.47 | -8% | 0.7634 | -| 1d || 29.1 | 34.6 | +19% || 0.51 | 0.47 | -8% | 0.4480 | +| 20a || 586.2 | 558.8 | -5% || 0.51 | 0.47 | -8% | 0.5001 | -| 20b || 427.7 | 428.8 | +0% || 0.51 | 0.47 | -8% | 0.9776 | -| 20c || 383.6 | 398.3 | +4% || 0.51 | 0.47 | -8% | 0.6379 | -| 21a || 68.2 | 86.6 | +27% || 0.51 | 0.47 | -8% | 0.1931 | -| 21b || 59.3 | 64.3 | +8% || 0.51 | 0.47 | -8% | 0.5503 | -| 21c || 65.4 | 93.1 | +42% || 0.51 | 0.47 | -8% | 0.0506 | -| 22a || 473.8 | 585.0 | +23% || 0.51 | 0.47 | -8% | 0.0056 | -| 22b || 345.7 | 360.8 | +4% || 0.51 | 0.47 | -8% | 0.6824 | -| 22c || 861.8 | 924.3 | +7% || 0.51 | 0.47 | -8% | 0.2341 | -| 22d || 1152.0 | 1342.4 | +17% || 0.51 | 0.47 | -8% | 0.0003 | -| 23a || 164.1 | 210.2 | +28% || 0.51 | 0.47 | -8% | 0.1370 | -| 23b || 109.9 | 131.8 | +20% || 0.51 | 0.47 | -8% | 0.2541 | -| 23c || 187.0 | 186.7 | -0% || 0.51 | 0.47 | -8% | 0.9924 | +| 24a || 252.1 | 238.0 | -6% || 0.51 | 0.47 | -8% | 0.6489 | -| 24b || 155.8 | 176.2 | +13% || 0.51 | 0.47 | -8% | 0.4008 | -| 25a || 440.7 | 559.3 | +27% || 0.51 | 0.47 | -8% | 0.0048 | -| 25b || 123.7 | 153.2 | +24% || 0.51 | 0.47 | -8% | 0.1714 | -| 25c || 937.7 | 999.2 | +7% || 0.51 | 0.47 | -8% | 0.2415 | +| 26a || 356.4 | 310.6 | -13% || 0.51 | 0.47 | -8% | 0.1771 | -| 26b || 182.2 | 247.2 | +36% || 0.51 | 0.47 | -8% | 0.0578 | -| 26c || 407.1 | 491.2 | +21% || 0.51 | 0.47 | -8% | 0.0270 | -| 27a || 92.4 | 111.5 | +21% || 0.51 | 0.47 | -8% | 0.3296 | +| 27b || 120.4 | 86.6 | -28% || 0.51 | 0.47 | -8% | 0.1396 | +| 27c || 76.0 | 71.2 | -6% || 0.51 | 0.47 | -8% | 0.7112 | -| 28a || 620.4 | 719.9 | +16% || 0.51 | 0.47 | -8% | 0.0172 | -| 28b || 201.1 | 201.7 | +0% || 0.51 | 0.47 | -8% | 0.9835 | -| 28c || 749.7 | 749.4 | -0% || 0.51 | 0.47 | -8% | 0.9949 | -| 29a || 214.5 | 221.6 | +3% || 0.51 | 0.47 | -8% | 0.7727 | -| 29b || 137.2 | 135.2 | -1% || 0.51 | 0.47 | -8% | 0.9224 | -| 29c || 209.3 | 263.8 | +26% || 0.51 | 0.47 | -8% | 0.0496 | +| 2a || 190.7 | 176.9 | -7% || 0.51 | 0.47 | -8% | 0.6728 | -| 2b || 146.2 | 155.5 | +6% || 0.51 | 0.47 | -8% | 0.7371 | -| 2c || 57.7 | 81.9 | +42% || 0.51 | 0.47 | -8% | 0.0236 | +| 2d || 230.5 | 169.7 | -26% || 0.51 | 0.47 | -8% | 0.0150 | -| 30a || 301.1 | 323.3 | +7% || 0.51 | 0.47 | -8% | 0.4791 | -| 30b || 262.6 | 279.5 | +6% || 0.51 | 0.47 | -8% | 0.6247 | -| 30c || 743.6 | 716.0 | -4% || 0.51 | 0.47 | -8% | 0.6306 | -| 31a || 265.5 | 358.8 | +35% || 0.51 | 0.47 | -8% | 0.0078 | -| 31b || 163.6 | 164.5 | +1% || 0.51 | 0.47 | -8% | 0.9688 | -| 31c || 331.2 | 378.2 | +14% || 0.51 | 0.47 | -8% | 0.2368 | +| 32a || 45.7 | 36.9 | -19% || 0.51 | 0.47 | -8% | 0.3864 | -| 32b || 109.1 | 119.6 | +10% || 0.51 | 0.47 | -8% | 0.4269 | -| 33a || 62.5 | 95.0 | +52% || 0.51 | 0.47 | -8% | 0.0912 | -| 33b || 48.3 | 71.5 | +48% || 0.51 | 0.47 | -8% | 0.0362 | -| 33c || 69.3 | 69.4 | +0% || 0.51 | 0.47 | -8% | 0.9914 | +| 3a || 411.3 | 351.9 | -14% || 0.51 | 0.47 | -8% | 0.1419 | -| 3b || 67.6 | 71.7 | +6% || 0.51 | 0.47 | -8% | 0.8071 | -| 3c || 520.7 | 608.5 | +17% || 0.51 | 0.47 | -7% | 0.0601 | -| 4a || 189.0 | 236.4 | +25% || 0.51 | 0.47 | -8% | 0.0855 | +| 4b || 49.0 | 37.2 | -24% || 0.51 | 0.47 | -8% | 0.2436 | +| 4c || 236.9 | 220.2 | -7% || 0.51 | 0.47 | -8% | 0.5775 | -| 5a || 158.0 | 178.7 | +13% || 0.51 | 0.47 | -8% | 0.3820 | -| 5b || 174.7 | 218.9 | +25% || 0.51 | 0.47 | -8% | 0.0789 | -| 5c || 233.2 | 256.3 | +10% || 0.51 | 0.47 | -8% | 0.4364 | -| 6a || 91.3 | 110.6 | +21% || 0.51 | 0.47 | -8% | 0.2021 | -| 6b || 140.2 | 134.6 | -4% || 0.51 | 0.47 | -8% | 0.7289 | -| 6c || 91.7 | 110.5 | +21% || 0.51 | 0.47 | -8% | 0.2827 | -| 6d || 530.5 | 534.2 | +1% || 0.51 | 0.47 | -8% | 0.9192 | -| 6e || 89.8 | 100.4 | +12% || 0.51 | 0.47 | -8% | 0.4221 | -| 6f || 927.6 | 957.8 | +3% || 0.51 | 0.47 | -8% | 0.4386 | -| 7a || 115.3 | 112.8 | -2% || 0.51 | 0.47 | -8% | 0.9027 | -| 7b || 106.7 | 146.6 | +37% || 0.51 | 0.47 | -8% | 0.0887 | -| 7c || 801.6 | 905.5 | +13% || 0.51 | 0.47 | -8% | 0.0237 | -| 8a || 115.8 | 137.1 | +18% || 0.51 | 0.47 | -8% | 0.3208 | -| 8b || 156.6 | 172.5 | +10% || 0.51 | 0.47 | -8% | 0.6300 | -| 8c || 2166.8 | 2388.6 | +10% || 0.51 | 0.47 | -8% | 0.0003 | -| 8d || 641.5 | 672.3 | +5% || 0.51 | 0.47 | -8% | 0.5184 | +| 9a || 527.4 | 491.6 | -7% || 0.51 | 0.47 | -8% | 0.5015 | -| 9b || 321.8 | 334.0 | +4% || 0.51 | 0.47 | -8% | 0.7184 | -| 9c || 596.1 | 707.9 | +19% || 0.51 | 0.47 | -8% | 0.0183 | -| 9d || 948.9 | 934.2 | -2% || 0.51 | 0.47 | -8% | 0.7713 | +---------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 50552.4 | 54323.7 | +7% || | | | | -| Geomean || | | || | | -8% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

[mweisgut]

[6] Master vs. Memory Tracking Disabled (Iter. 2)

Command:

../scripts/benchmark_all.sh 1c24c5f49719e589324037c5037866a7388a034c 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8

System

nemea - click to expand

| property | value | | -- | -- | | Hostname | nemea | | CPU | Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz | | Memory | 1.2TB | | numactl | nodebind: 1 | | numactl | membind: 1 |

Commit Info and Build Time	commit	date	message	build time
1c24c5f49	21.07.2022 21:52	Ensure correct results of EXTRACT expressions (#2482)	real 333.27 user 2773.77 sys 79.86
5acb2f2a0	22.07.2022 10:35	disabled tracking	real 329.67 user 2811.49 sys 80.18

hyriseBenchmarkTPCH - single-threaded, SF 10.0

Sum of avg. item runtimes: +7% || Geometric mean of throughput changes: -5%

Configuration Overview - click to expand

```diff +Configuration Overview----+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 06:17:41 | 2022-07-25 20:45:22 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 5719.0 | 5751.2 | +1% || 0.17 | 0.17 | -1% | 0.7154 | | TPC-H 02 || 43.4 | 42.8 | -1%˄ || 23.04 | 23.37 | +1%˄ | 0.3115 | -| TPC-H 03 || 2227.8 | 2755.4 | +24% || 0.45 | 0.36 | -19% | 0.0000 | -| TPC-H 04 || 1397.4 | 1637.1 | +17% || 0.72 | 0.61 | -15% | 0.0000 | -| TPC-H 05 || 3123.6 | 3475.1 | +11% || 0.32 | 0.29 | -10% | 0.0000 | | TPC-H 06 || 153.6 | 154.2 | +0%˄ || 6.51 | 6.49 | -0%˄ | 0.6041 | -| TPC-H 07 || 928.2 | 1031.7 | +11% || 1.08 | 0.97 | -10% | 0.0000 | -| TPC-H 08 || 705.3 | 783.3 | +11% || 1.42 | 1.28 | -10% | 0.0000 | | TPC-H 09 || 5221.9 | 5331.8 | +2% || 0.19 | 0.19 | -2% | 0.0000 | -| TPC-H 10 || 2964.3 | 3154.0 | +6% || 0.34 | 0.32 | -6% | 0.0000 | | TPC-H 11 || 73.4 | 73.2 | -0%˄ || 13.61 | 13.65 | +0%˄ | 0.5308 | -| TPC-H 12 || 989.9 | 1071.0 | +8% || 1.01 | 0.93 | -8% | 0.0000 | -| TPC-H 13 || 4693.6 | 4929.1 | +5% || 0.21 | 0.20 | -5% | 0.0000 | | TPC-H 14 || 420.9 | 429.9 | +2%˄ || 2.38 | 2.33 | -2%˄ | 0.0000 | | TPC-H 15 || 190.4 | 190.4 | +0%˄ || 5.25 | 5.25 | -0%˄ | 0.7974 | | TPC-H 16 || 631.8 | 633.0 | +0% || 1.58 | 1.58 | -0% | 0.2848 | | TPC-H 17 || 215.2 | 214.3 | -0%˄ || 4.65 | 4.67 | +0%˄ | 0.0000 | | TPC-H 18 || 1586.5 | 1579.4 | -0% || 0.63 | 0.63 | +0% | 0.5894 | | TPC-H 19 || 268.8 | 266.8 | -1%˄ || 3.72 | 3.75 | +1%˄ | 0.1475 | | TPC-H 20 || 400.2 | 396.8 | -1%˄ || 2.50 | 2.52 | +1%˄ | 0.0053 | -| TPC-H 21 || 4719.4 | 5363.5 | +14% || 0.21 | 0.19 | -12% | 0.0000 | -| TPC-H 22 || 426.1 | 477.7 | +12%˄ || 2.35 | 2.09 | -11%˄ | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 37100.7 | 39741.7 | +7% || | | | | -| Geomean || | | || | | -5% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - single-threaded, SF 0.01

Sum of avg. item runtimes: -4% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview----+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_st_s01.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st_s01.json | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 06:37:01 | 2022-07-25 21:04:58 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | scale_factor | 0.009999999776482582 | 0.009999999776482582 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +--------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ | TPC-H 01 || 4.8 | 4.8 | -0%˄ || 209.53 | 210.09 | +0%˄ | 0.3431 | | TPC-H 02 || 3.8 | 3.8 | +1%˄ || 265.42 | 263.91 | -1%˄ | 0.9224 | | TPC-H 03 || 0.6 | 0.7 | +2%˄ || 1558.18 | 1524.33 | -2%˄ | 0.0593 | | TPC-H 04 || 0.4 | 0.4 | +1%˄ || 2348.40 | 2315.03 | -1%˄ | 0.0013 | | TPC-H 05 || 1.0 | 1.0 | +2%˄ || 1012.76 | 995.15 | -2%˄ | 0.0155 | | TPC-H 06 || 0.2 | 0.2 | +0%˄ || 5698.95 | 5690.64 | -0%˄ | 0.7998 | +| TPC-H 07 || 11.4 | 9.7 | -15%˄ || 87.64 | 102.61 | +17%˄ | 0.0219 | | TPC-H 08 || 15.4 | 15.4 | -0%˄ || 65.08 | 65.10 | +0%˄ | 0.9887 | +| TPC-H 09 || 3.8 | 3.3 | -11%˄ || 265.69 | 299.09 | +13%˄ | 0.4281 | | TPC-H 10 || 0.8 | 0.8 | +0%˄ || 1286.16 | 1281.78 | -0%˄ | 0.4661 | | TPC-H 11 || 0.2 | 0.2 | -1%˄ || 4784.21 | 4831.30 | +1%˄ | 0.4730 | | TPC-H 12 || 0.6 | 0.6 | -2%˄ || 1607.29 | 1642.84 | +2%˄ | 0.0000 | | TPC-H 13 || 2.0 | 1.9 | -4%˄ || 500.28 | 521.70 | +4%˄ | 0.0000 | | TPC-H 14 || 0.3 | 0.3 | -0%˄ || 2944.02 | 2958.25 | +0%˄ | 0.6241 | | TPC-H 15 || 1.1 | 1.1 | +1%˄ || 893.19 | 883.95 | -1%˄ | 0.0000 | | TPC-H 16 || 1.9 | 1.9 | -0%˄ || 518.28 | 519.61 | +0%˄ | 0.3694 | -| TPC-H 17 || 0.6 | 0.7 | +8%˄ || 1536.66 | 1421.72 | -7%˄ | 0.5125 | | TPC-H 18 || 1.1 | 1.1 | -0%˄ || 883.86 | 885.14 | +0%˄ | 0.2179 | | TPC-H 19 || 4.8 | 4.8 | -0%˄ || 209.29 | 209.57 | +0%˄ | 0.1684 | | TPC-H 20 || 2.0 | 2.0 | -1%˄ || 494.87 | 500.96 | +1%˄ | 0.6880 | | TPC-H 21 || 1.4 | 1.4 | -3%˄ || 699.80 | 719.60 | +3%˄ | 0.5335 | | TPC-H 22 || 1.0 | 1.0 | +0%˄ || 1010.76 | 1009.67 | -0%˄ | 0.6338 | +----------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 59.3 | 57.1 | -4% || | | | | | Geomean || | | || | | +1% | | +----------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 28 cores, SF 10.0

Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -3%

Configuration Overview - click to expand

```diff +Configuration Overview---------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt_ordered.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt_ordered.json | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 06:37:29 | 2022-07-25 21:05:26 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | TPC-H 01 || 4748.2 | 4768.9 | +0% || 0.20 | 0.20 | +0% | (run time too short) | | TPC-H 02 || 63.0 | 62.6 | -1% || 14.55 | 14.63 | +1% | 0.5466 | -| TPC-H 03 || 1120.2 | 1204.1 | +7% || 0.88 | 0.82 | -8% | 0.0000 | -| TPC-H 04 || 595.7 | 664.7 | +12% || 1.65 | 1.48 | -10% | (run time too short) | -| TPC-H 05 || 1022.1 | 1066.7 | +4% || 0.97 | 0.92 | -5% | (run time too short) | | TPC-H 06 || 72.1 | 71.6 | -1% || 12.95 | 13.03 | +1% | 0.0316 | | TPC-H 07 || 462.8 | 474.8 | +3% || 2.13 | 2.08 | -2% | 0.0000 | -| TPC-H 08 || 401.9 | 431.2 | +7% || 2.45 | 2.28 | -7% | 0.0000 | | TPC-H 09 || 2531.8 | 2522.4 | -0% || 0.38 | 0.38 | +0% | (run time too short) | -| TPC-H 10 || 1599.6 | 1676.1 | +5% || 0.62 | 0.58 | -5% | (run time too short) | | TPC-H 11 || 90.2 | 91.0 | +1% || 10.50 | 10.42 | -1% | 0.1655 | -| TPC-H 12 || 550.9 | 586.2 | +6% || 1.78 | 1.68 | -6% | 0.0000 | | TPC-H 13 || 3005.3 | 3068.6 | +2% || 0.32 | 0.32 | +0% | (run time too short) | | TPC-H 14 || 171.6 | 175.3 | +2% || 5.65 | 5.53 | -2% | 0.0001 | | TPC-H 15 || 169.8 | 168.1 | -1% || 5.70 | 5.78 | +1% | 0.0617 | | TPC-H 16 || 775.2 | 775.6 | +0% || 1.27 | 1.27 | -0% | (run time too short) | | TPC-H 17 || 93.6 | 93.8 | +0% || 10.15 | 10.12 | -0% | 0.6920 | -| TPC-H 18 || 2455.3 | 2597.5 | +6% || 0.40 | 0.38 | -4% | (run time too short) | | TPC-H 19 || 156.7 | 155.5 | -1% || 6.17 | 6.22 | +1% | 0.2233 | | TPC-H 20 || 254.7 | 248.9 | -2% || 3.83 | 3.93 | +3% | 0.0008 | -| TPC-H 21 || 1201.4 | 1332.7 | +11% || 0.82 | 0.73 | -10% | (run time too short) | -| TPC-H 22 || 153.6 | 173.5 | +13% || 6.28 | 5.58 | -11% | 0.0000 | +----------++----------+---------+--------++----------+----------+--------+----------------------+ | Sum || 21695.7 | 22409.7 | +3% || | | | | | Geomean || | | || | | -3% | | +----------++----------+---------+--------++----------+----------+--------+----------------------+ ```

hyriseBenchmarkTPCH - multi-threaded, shuffled, 28 clients, 28 cores, SF 10.0

Sum of avg. item runtimes: +9% || Geometric mean of throughput changes: -9%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCH_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | clustering | None | None | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 07:01:13 | 2022-07-25 21:29:14 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 10.0 | 10.0 | | time_unit | ns | ns | | use_prepared_statements | False | False | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +----------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +----------++----------+---------+--------++----------+----------+--------+---------+ -| TPC-H 01 || 5506.5 | 5690.4 | +3% || 0.47 | 0.42 | -9% | 0.0232 | -| TPC-H 02 || 343.7 | 365.9 | +6% || 0.47 | 0.43 | -9% | 0.7521 | -| TPC-H 03 || 3434.3 | 3996.2 | +16% || 0.47 | 0.43 | -9% | 0.0048 | -| TPC-H 04 || 2574.0 | 2713.7 | +5% || 0.47 | 0.43 | -9% | 0.4436 | -| TPC-H 05 || 4014.3 | 4830.3 | +20% || 0.47 | 0.43 | -9% | 0.0004 | -| TPC-H 06 || 462.6 | 487.3 | +5% || 0.47 | 0.43 | -9% | 0.7232 | -| TPC-H 07 || 2434.9 | 2722.6 | +12% || 0.47 | 0.43 | -9% | 0.0975 | -| TPC-H 08 || 1866.3 | 2025.9 | +9% || 0.47 | 0.43 | -9% | 0.3028 | -| TPC-H 09 || 5874.3 | 6393.2 | +9% || 0.47 | 0.42 | -9% | 0.0258 | -| TPC-H 10 || 4686.0 | 5067.0 | +8% || 0.47 | 0.43 | -9% | 0.0668 | -| TPC-H 11 || 546.9 | 665.8 | +22% || 0.47 | 0.43 | -9% | 0.2488 | -| TPC-H 12 || 2113.7 | 2442.8 | +16% || 0.47 | 0.43 | -9% | 0.0388 | -| TPC-H 13 || 5660.5 | 5962.4 | +5% || 0.47 | 0.42 | -9% | 0.0040 | -| TPC-H 14 || 1054.6 | 1194.6 | +13% || 0.47 | 0.43 | -9% | 0.2119 | +| TPC-H 15 || 548.8 | 470.1 | -14% || 0.47 | 0.43 | -9% | 0.2315 | -| TPC-H 16 || 2267.7 | 2349.1 | +4% || 0.47 | 0.43 | -9% | 0.6200 | -| TPC-H 17 || 494.6 | 608.2 | +23% || 0.47 | 0.43 | -9% | 0.1162 | -| TPC-H 18 || 3510.3 | 3520.0 | +0% || 0.47 | 0.43 | -9% | 0.9193 | -| TPC-H 19 || 722.9 | 882.6 | +22% || 0.47 | 0.43 | -9% | 0.1035 | -| TPC-H 20 || 1168.6 | 1337.1 | +14% || 0.47 | 0.43 | -9% | 0.0976 | -| TPC-H 21 || 5697.1 | 5930.9 | +4% || 0.47 | 0.42 | -9% | 0.3638 | -| TPC-H 22 || 748.1 | 1012.9 | +35% || 0.47 | 0.43 | -9% | 0.0118 | +----------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 55730.7 | 60669.0 | +9% || | | | | -| Geomean || | | || | | -9% | | +----------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - single-threaded

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 07:22:56 | 2022-07-25 21:50:59 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 258.4 | 261.8 | +1%˄ || 3.87 | 3.82 | -1%˄ | 0.0014 | | 03 || 69.1 | 69.4 | +0%˄ || 14.47 | 14.42 | -0%˄ | 0.0988 | | 06 || 151.1 | 150.2 | -1%˄ || 6.62 | 6.66 | +1%˄ | 0.0157 | | 07 || 297.7 | 295.7 | -1%˄ || 3.36 | 3.38 | +1%˄ | 0.3740 | | 09 || 631.7 | 636.3 | +1% || 1.58 | 1.57 | -1% | 0.0000 | | 10 || 131.9 | 133.6 | +1%˄ || 7.58 | 7.49 | -1%˄ | 0.0850 | | 13 || 422.9 | 421.6 | -0%˄ || 2.36 | 2.37 | +0%˄ | 0.4942 | -| 15 || 102.6 | 108.1 | +5%˄ || 9.75 | 9.25 | -5%˄ | 0.0000 | +| 16 || 152.9 | 143.7 | -6%˄ || 6.54 | 6.96 | +6%˄ | 0.0000 | | 17 || 306.2 | 309.6 | +1%˄ || 3.27 | 3.23 | -1%˄ | 0.0000 | | 19 || 99.8 | 99.7 | -0%˄ || 10.02 | 10.03 | +0%˄ | 0.9274 | | 25 || 163.2 | 167.4 | +3%˄ || 6.13 | 5.97 | -3%˄ | 0.0000 | | 26 || 130.1 | 133.5 | +3%˄ || 7.69 | 7.49 | -3%˄ | 0.0000 | | 28 || 603.1 | 595.5 | -1%˄ || 1.66 | 1.68 | +1%˄ | 0.0000 | | 29 || 468.6 | 474.9 | +1%˄ || 2.13 | 2.11 | -1%˄ | 0.0000 | | 31 || 1262.8 | 1249.6 | -1% || 0.79 | 0.80 | +1% | 0.0000 | -| 32 || 40.7 | 44.0 | +8%˄ || 24.56 | 22.74 | -7%˄ | 0.0000 | | 34 || 157.6 | 163.8 | +4%˄ || 6.35 | 6.11 | -4%˄ | 0.0000 | -| 35 || 605.7 | 642.2 | +6%˄ || 1.65 | 1.56 | -6%˄ | 0.0000 | | 37 || 274.9 | 267.2 | -3%˄ || 3.64 | 3.74 | +3%˄ | 0.0002 | | 39a || 1751.2 | 1678.8 | -4% || 0.57 | 0.60 | +4% | 0.0000 | | 39b || 1701.8 | 1711.1 | +1% || 0.59 | 0.58 | -1% | 0.4463 | | 41 || 288.8 | 287.0 | -1%˄ || 3.46 | 3.48 | +1%˄ | 0.0000 | +| 42 || 81.4 | 77.4 | -5%˄ || 12.28 | 12.93 | +5%˄ | 0.0000 | | 43 || 961.6 | 934.0 | -3% || 1.04 | 1.07 | +3% | 0.0000 | | 45 || 113.5 | 115.2 | +1%˄ || 8.81 | 8.68 | -1%˄ | 0.0000 | | 48 || 1011.2 | 1025.0 | +1% || 0.99 | 0.98 | -1% | 0.0026 | | 50 || 108.1 | 108.7 | +1%˄ || 9.25 | 9.20 | -1%˄ | 0.0000 | | 52 || 78.3 | 77.4 | -1%˄ || 12.77 | 12.91 | +1%˄ | 0.0000 | | 55 || 75.0 | 74.3 | -1%˄ || 13.32 | 13.45 | +1%˄ | 0.0000 | | 62 || 542.2 | 525.1 | -3%˄ || 1.84 | 1.90 | +3%˄ | 0.0000 | | 65 || 1757.9 | 1733.5 | -1% || 0.57 | 0.58 | +1% | 0.0068 | | 69 || 119.2 | 120.6 | +1%˄ || 8.39 | 8.29 | -1%˄ | 0.0000 | | 73 || 76.6 | 76.1 | -1%˄ || 13.05 | 13.14 | +1%˄ | 0.0009 | | 79 || 467.0 | 465.0 | -0%˄ || 2.14 | 2.15 | +0%˄ | 0.0000 | | 81 || 206.5 | 201.9 | -2%˄ || 4.84 | 4.95 | +2%˄ | 0.0000 | | 82 || 355.1 | 354.5 | -0%˄ || 2.82 | 2.82 | +0%˄ | 0.7945 | | 83 || 39.9 | 39.6 | -1%˄ || 25.05 | 25.22 | +1%˄ | 0.5332 | +| 85 || 133.9 | 124.6 | -7%˄ || 7.47 | 8.03 | +7%˄ | 0.0000 | -| 88 || 615.5 | 657.6 | +7% || 1.62 | 1.52 | -6% | 0.0000 | -| 91 || 15.3 | 16.1 | +5%˄ || 65.16 | 62.00 | -5%˄ | 0.0000 | | 92 || 34.5 | 35.6 | +3%˄ || 28.99 | 28.09 | -3%˄ | 0.0000 | | 93 || 3862.6 | 3821.8 | -1% || 0.26 | 0.26 | +1% | 0.0024 | | 94 || 97.7 | 94.4 | -3%˄ || 10.23 | 10.60 | +4%˄ | 0.0000 | | 95 || 9072.9 | 9017.9 | -1% || 0.11 | 0.11 | +1% | ˅ | | 96 || 56.9 | 57.2 | +0%˄ || 17.56 | 17.49 | -0%˄ | 0.5184 | | 97 || 3255.5 | 3287.6 | +1% || 0.31 | 0.30 | -1% | 0.2596 | | 99 || 1043.2 | 1018.1 | -2% || 0.96 | 0.98 | +2% | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 34254.4 | 34103.8 | -0% || | | | | | Geomean || | | || | | -0% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | | || ˅ Insufficient number of runs for p-value calculation | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCDS - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCDS_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 07:48:56 | 2022-07-25 22:16:55 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 01 || 632.8 | 635.3 | +0% || 0.47 | 0.47 | +0% | 0.9701 | -| 03 || 157.3 | 238.8 | +52% || 0.47 | 0.47 | -0% | 0.0413 | +| 06 || 707.8 | 587.1 | -17% || 0.47 | 0.47 | -0% | 0.1695 | | 07 || 862.1 | 824.7 | -4% || 0.47 | 0.47 | -0% | 0.6721 | | 09 || 1292.9 | 1285.1 | -1% || 0.47 | 0.47 | -0% | 0.9229 | -| 10 || 412.1 | 477.0 | +16% || 0.47 | 0.47 | -1% | 0.4256 | +| 13 || 1615.1 | 1518.2 | -6% || 0.47 | 0.47 | +0% | 0.4923 | -| 15 || 404.2 | 504.6 | +25% || 0.47 | 0.47 | -0% | 0.2008 | +| 16 || 826.2 | 648.2 | -22% || 0.47 | 0.47 | -0% | 0.0981 | -| 17 || 884.6 | 1104.7 | +25% || 0.47 | 0.47 | -0% | 0.0500 | +| 19 || 532.5 | 439.7 | -17% || 0.47 | 0.47 | -0% | 0.2979 | +| 25 || 722.1 | 666.0 | -8% || 0.47 | 0.47 | +0% | 0.5364 | | 26 || 491.2 | 512.5 | +4% || 0.47 | 0.47 | -0% | 0.7687 | -| 28 || 530.2 | 570.3 | +8% || 0.47 | 0.47 | -0% | 0.5708 | -| 29 || 1275.6 | 1340.3 | +5% || 0.47 | 0.47 | -0% | 0.5572 | | 31 || 1919.4 | 1886.1 | -2% || 0.47 | 0.47 | -0% | 0.8181 | +| 32 || 333.7 | 212.1 | -36% || 0.47 | 0.47 | -0% | 0.0268 | +| 34 || 461.1 | 434.3 | -6% || 0.47 | 0.47 | -0% | 0.6336 | -| 35 || 1781.8 | 2053.5 | +15% || 0.47 | 0.47 | +0% | 0.0694 | -| 37 || 501.8 | 551.3 | +10% || 0.47 | 0.47 | -0% | 0.4858 | | 39a || 2086.2 | 2175.4 | +4% || 0.47 | 0.47 | -0% | 0.4459 | | 39b || 2145.1 | 2162.0 | +1% || 0.47 | 0.47 | -0% | 0.8867 | | 41 || 2973.2 | 2860.0 | -4% || 0.47 | 0.47 | -0% | 0.5168 | | 42 || 269.8 | 281.7 | +4% || 0.47 | 0.47 | -0% | 0.7994 | -| 43 || 1284.9 | 1370.6 | +7% || 0.47 | 0.47 | -0% | 0.3079 | | 45 || 527.7 | 530.1 | +0% || 0.47 | 0.47 | -0% | 0.9758 | | 48 || 2544.0 | 2548.8 | +0% || 0.47 | 0.47 | +0% | 0.9761 | | 50 || 504.0 | 482.0 | -4% || 0.47 | 0.47 | -0% | 0.7199 | | 52 || 342.2 | 342.9 | +0% || 0.47 | 0.47 | -0% | 0.9902 | -| 55 || 247.1 | 270.0 | +9% || 0.47 | 0.47 | -0% | 0.6312 | -| 62 || 839.7 | 893.0 | +6% || 0.47 | 0.47 | -0% | 0.4936 | | 65 || 3586.8 | 3701.7 | +3% || 0.47 | 0.47 | -0% | 0.2069 | -| 69 || 514.2 | 598.7 | +16% || 0.47 | 0.47 | -0% | 0.3294 | +| 73 || 298.3 | 271.1 | -9% || 0.47 | 0.47 | +0% | 0.6534 | -| 79 || 1114.7 | 1166.3 | +5% || 0.47 | 0.47 | -0% | 0.5186 | | 81 || 787.6 | 769.6 | -2% || 0.47 | 0.47 | +0% | 0.8421 | -| 82 || 752.0 | 931.1 | +24% || 0.47 | 0.47 | -0% | 0.1144 | +| 83 || 255.2 | 232.3 | -9% || 0.47 | 0.47 | -0% | 0.5784 | | 85 || 685.4 | 659.7 | -4% || 0.47 | 0.47 | -0% | 0.7321 | | 88 || 752.5 | 776.5 | +3% || 0.47 | 0.47 | +0% | 0.7861 | -| 91 || 162.9 | 222.6 | +37% || 0.47 | 0.47 | -0% | 0.3419 | +| 92 || 257.5 | 234.5 | -9% || 0.47 | 0.47 | +0% | 0.6470 | | 93 || 2417.5 | 2380.0 | -2% || 0.47 | 0.47 | -0% | 0.7703 | +| 94 || 790.1 | 670.2 | -15% || 0.47 | 0.47 | -0% | 0.2568 | | 95 || 8050.9 | 7875.2 | -2% || 0.47 | 0.47 | +0% | 0.3235 | +| 96 || 209.0 | 176.8 | -15% || 0.47 | 0.47 | -0% | 0.3562 | | 97 || 4343.2 | 4412.1 | +2% || 0.47 | 0.47 | -0% | 0.6764 | +| 99 || 1386.1 | 1261.6 | -9% || 0.47 | 0.47 | -0% | 0.1290 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 56472.4 | 56746.4 | +0% || | | | | | Geomean || | | || | | -0% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - single-threaded

Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -0%

Configuration Overview - click to expand

```diff +Configuration Overview---+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 08:09:25 | 2022-07-25 22:37:23 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 22.8 | 23.1 | +1% || 4.25 | 4.25 | +0% | 0.0116 | | New-Order || 16.4 | 16.4 | -0% || 47.94 | 47.92 | -0% | 0.9854 | | Order-Status || 1.1 | 1.1 | +0% || 4.27 | 4.27 | +0% | 0.6543 | | Payment || 2.1 | 2.0 | -1% || 45.82 | 45.72 | -0% | 0.0832 | | Stock-Level || 3.5 | 3.6 | +1% || 4.27 | 4.27 | +0% | 0.1032 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 45.9 | 46.2 | +1% || | | | | | Geomean || | | || | | -0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkTPCC - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +0%

Configuration Overview - click to expand

```diff +Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkTPCC_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 08:10:26 | 2022-07-25 22:38:24 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | scale_factor | 1 | 1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +--------------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Delivery || 218.7 | 215.7 | -1% || 4.34 | 4.39 | +1% | 0.0643 | | unsucc.: || 4.4 | 4.3 | -0% || 62.89 | 62.89 | -0% | | | New-Order || 57.2 | 57.0 | -0% || 133.48 | 134.55 | +1% | 0.2418 | | unsucc.: || 5.0 | 4.9 | -2% || 622.86 | 622.35 | -0% | | | Order-Status || 7.5 | 7.6 | +0% || 67.23 | 67.28 | +0% | 0.3749 | | Payment || 9.9 | 9.9 | +0% || 21.43 | 21.47 | +0% | 0.5650 | | unsucc.: || 4.4 | 4.4 | -0% || 701.30 | 701.80 | +0% | | | Stock-Level || 17.3 | 17.3 | -0% || 67.23 | 67.28 | +0% | 0.3435 | +--------------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 310.5 | 307.5 | -1% || | | | | | Geomean || | | || | | +0% | | +--------------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - single-threaded

Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +1%

Configuration Overview - click to expand

```diff +Configuration Overview---+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_st.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_st.json | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Ordered | Ordered | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 1 | 1 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 0 | 0 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 08:30:34 | 2022-07-25 22:58:31 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 60000000000 | 60000000000 | | max_runs | 100 | 100 | | time_unit | ns | ns | | using_scheduler | False | False | | verify | False | False | | warmup_duration | 1000000000 | 1000000000 | +-------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ | 10a || 168.1 | 164.2 | -2%˄ || 5.95 | 6.09 | +2%˄ | 0.0000 | | 10b || 116.1 | 114.7 | -1%˄ || 8.61 | 8.72 | +1%˄ | 0.0000 | | 10c || 323.1 | 315.3 | -2%˄ || 3.09 | 3.17 | +2%˄ | 0.0000 | | 11a || 15.2 | 15.1 | -1%˄ || 65.90 | 66.37 | +1%˄ | 0.0000 | | 11b || 15.0 | 14.8 | -1%˄ || 66.65 | 67.33 | +1%˄ | 0.0000 | | 11c || 20.6 | 20.2 | -2%˄ || 48.47 | 49.45 | +2%˄ | 0.0000 | | 11d || 23.6 | 23.0 | -3%˄ || 42.33 | 43.56 | +3%˄ | 0.0000 | +| 12a || 48.7 | 46.4 | -5%˄ || 20.52 | 21.54 | +5%˄ | 0.0000 | | 12b || 29.3 | 29.1 | -1%˄ || 34.16 | 34.38 | +1%˄ | 0.0000 | | 12c || 83.4 | 83.2 | -0%˄ || 11.99 | 12.02 | +0%˄ | 0.2705 | | 13a || 277.0 | 268.5 | -3%˄ || 3.61 | 3.72 | +3%˄ | 0.0000 | | 13b || 154.2 | 155.0 | +1%˄ || 6.48 | 6.45 | -1%˄ | 0.0000 | | 13c || 115.7 | 118.3 | +2%˄ || 8.64 | 8.45 | -2%˄ | 0.0000 | +| 13d || 589.4 | 556.7 | -6%˄ || 1.70 | 1.80 | +6%˄ | 0.0000 | | 14a || 415.8 | 412.8 | -1%˄ || 2.40 | 2.42 | +1%˄ | 0.0006 | -| 14b || 233.6 | 245.7 | +5%˄ || 4.28 | 4.07 | -5%˄ | 0.0000 | | 14c || 491.2 | 491.2 | +0%˄ || 2.04 | 2.04 | -0%˄ | 0.9986 | | 15a || 81.6 | 79.1 | -3%˄ || 12.26 | 12.65 | +3%˄ | 0.0000 | | 15b || 80.8 | 78.5 | -3%˄ || 12.38 | 12.75 | +3%˄ | 0.0000 | | 15c || 85.8 | 83.6 | -3%˄ || 11.65 | 11.96 | +3%˄ | 0.0000 | +| 15d || 83.7 | 79.8 | -5%˄ || 11.95 | 12.53 | +5%˄ | 0.0000 | | 16a || 2259.8 | 2292.2 | +1% || 0.44 | 0.44 | -1% | 0.0140 | | 16b || 3294.0 | 3327.8 | +1% || 0.30 | 0.30 | -1% | 0.0440 | | 16c || 2359.9 | 2372.2 | +1% || 0.42 | 0.42 | -1% | 0.2588 | | 16d || 2344.3 | 2338.1 | -0% || 0.43 | 0.43 | +0% | 0.4677 | | 17a || 585.4 | 596.8 | +2%˄ || 1.71 | 1.68 | -2%˄ | 0.0000 | | 17b || 460.8 | 474.1 | +3%˄ || 2.17 | 2.11 | -3%˄ | 0.0000 | | 17c || 441.0 | 451.9 | +2%˄ || 2.27 | 2.21 | -2%˄ | 0.0000 | | 17d || 522.1 | 530.8 | +2%˄ || 1.92 | 1.88 | -2%˄ | 0.0000 | | 17e || 1611.1 | 1587.1 | -1% || 0.62 | 0.63 | +2% | 0.0000 | | 17f || 947.3 | 954.9 | +1% || 1.06 | 1.05 | -1% | 0.0000 | -| 18a || 599.8 | 627.1 | +5%˄ || 1.67 | 1.59 | -4%˄ | 0.0000 | | 18b || 112.7 | 110.2 | -2%˄ || 8.87 | 9.07 | +2%˄ | 0.0000 | | 18c || 564.2 | 556.2 | -1%˄ || 1.77 | 1.80 | +1%˄ | 0.0000 | | 19a || 180.6 | 178.7 | -1%˄ || 5.54 | 5.60 | +1%˄ | 0.0000 | | 19b || 124.8 | 122.1 | -2%˄ || 8.01 | 8.19 | +2%˄ | 0.0000 | | 19c || 193.9 | 190.8 | -2%˄ || 5.16 | 5.24 | +2%˄ | 0.0000 | | 19d || 720.8 | 722.8 | +0% || 1.39 | 1.38 | -0% | 0.0002 | | 1a || 9.0 | 8.8 | -2%˄ || 111.54 | 113.46 | +2%˄ | 0.0000 | | 1b || 8.4 | 8.2 | -2%˄ || 119.22 | 121.37 | +2%˄ | 0.0000 | | 1c || 9.5 | 9.3 | -2%˄ || 105.47 | 107.98 | +2%˄ | 0.0000 | | 1d || 8.4 | 8.3 | -1%˄ || 119.33 | 120.88 | +1%˄ | 0.0000 | | 20a || 393.5 | 384.7 | -2%˄ || 2.54 | 2.60 | +2%˄ | 0.0000 | | 20b || 229.4 | 225.2 | -2%˄ || 4.36 | 4.44 | +2%˄ | 0.0000 | | 20c || 221.7 | 218.1 | -2%˄ || 4.51 | 4.59 | +2%˄ | 0.0000 | | 21a || 35.2 | 35.0 | -1%˄ || 28.37 | 28.60 | +1%˄ | 0.0000 | | 21b || 19.1 | 18.8 | -2%˄ || 52.29 | 53.23 | +2%˄ | 0.0000 | | 21c || 34.7 | 34.4 | -1%˄ || 28.79 | 29.05 | +1%˄ | 0.0000 | | 22a || 196.4 | 205.1 | +4%˄ || 5.09 | 4.88 | -4%˄ | 0.0000 | | 22b || 143.2 | 146.2 | +2%˄ || 6.98 | 6.84 | -2%˄ | 0.0000 | | 22c || 492.3 | 499.4 | +1%˄ || 2.03 | 2.00 | -1%˄ | 0.0000 | | 22d || 942.4 | 921.1 | -2% || 1.06 | 1.09 | +2% | 0.0000 | | 23a || 42.8 | 42.0 | -2%˄ || 23.34 | 23.81 | +2%˄ | 0.0000 | | 23b || 44.1 | 43.5 | -1%˄ || 22.66 | 23.00 | +1%˄ | 0.0000 | | 23c || 85.8 | 83.1 | -3%˄ || 11.66 | 12.03 | +3%˄ | 0.0000 | | 24a || 145.1 | 142.8 | -2%˄ || 6.89 | 7.00 | +2%˄ | 0.0000 | | 24b || 123.4 | 121.9 | -1%˄ || 8.11 | 8.20 | +1%˄ | 0.0000 | | 25a || 278.1 | 273.9 | -2%˄ || 3.60 | 3.65 | +2%˄ | 0.0000 | | 25b || 113.6 | 112.3 | -1%˄ || 8.81 | 8.90 | +1%˄ | 0.0000 | | 25c || 920.8 | 913.1 | -1% || 1.09 | 1.10 | +1% | 0.0000 | | 26a || 164.1 | 162.0 | -1%˄ || 6.09 | 6.17 | +1%˄ | 0.0000 | | 26b || 131.4 | 129.7 | -1%˄ || 7.61 | 7.71 | +1%˄ | 0.0000 | | 26c || 233.9 | 230.4 | -2%˄ || 4.27 | 4.34 | +2%˄ | 0.0000 | | 27a || 22.7 | 22.6 | -1%˄ || 43.99 | 44.25 | +1%˄ | 0.0289 | | 27b || 22.4 | 21.8 | -2%˄ || 44.73 | 45.79 | +2%˄ | 0.0000 | | 27c || 35.0 | 34.7 | -1%˄ || 28.55 | 28.85 | +1%˄ | 0.0000 | | 28a || 291.2 | 295.1 | +1%˄ || 3.43 | 3.39 | -1%˄ | 0.0000 | -| 28b || 37.2 | 48.9 | +31%˄ || 26.85 | 20.46 | -24%˄ | 0.0000 | | 28c || 329.3 | 335.3 | +2%˄ || 3.04 | 2.98 | -2%˄ | 0.0000 | | 29a || 101.8 | 101.5 | -0%˄ || 9.82 | 9.85 | +0%˄ | 0.0013 | | 29b || 76.4 | 75.9 | -1%˄ || 13.09 | 13.17 | +1%˄ | 0.0000 | | 29c || 149.5 | 147.4 | -1%˄ || 6.69 | 6.79 | +1%˄ | 0.0000 | | 2a || 46.4 | 45.6 | -2%˄ || 21.56 | 21.93 | +2%˄ | 0.0000 | | 2b || 37.9 | 37.4 | -1%˄ || 26.41 | 26.73 | +1%˄ | 0.0000 | | 2c || 21.9 | 22.0 | +0%˄ || 45.69 | 45.48 | -0%˄ | 0.0000 | | 2d || 65.3 | 64.7 | -1%˄ || 15.32 | 15.46 | +1%˄ | 0.0000 | | 30a || 150.1 | 147.5 | -2%˄ || 6.66 | 6.78 | +2%˄ | 0.0000 | | 30b || 128.1 | 126.0 | -2%˄ || 7.81 | 7.94 | +2%˄ | 0.0000 | | 30c || 382.7 | 380.4 | -1%˄ || 2.61 | 2.63 | +1%˄ | 0.0000 | | 31a || 146.5 | 144.8 | -1%˄ || 6.82 | 6.91 | +1%˄ | 0.0000 | | 31b || 125.0 | 123.9 | -1%˄ || 8.00 | 8.07 | +1%˄ | 0.0000 | | 31c || 170.4 | 168.9 | -1%˄ || 5.87 | 5.92 | +1%˄ | 0.0000 | | 32a || 12.9 | 12.8 | -1%˄ || 77.48 | 78.15 | +1%˄ | 0.0000 | -| 32b || 30.9 | 32.6 | +5%˄ || 32.34 | 30.68 | -5%˄ | 0.0000 | | 33a || 19.3 | 19.0 | -2%˄ || 51.70 | 52.56 | +2%˄ | 0.0000 | | 33b || 18.7 | 18.4 | -2%˄ || 53.43 | 54.30 | +2%˄ | 0.0000 | | 33c || 22.6 | 22.3 | -1%˄ || 44.30 | 44.89 | +1%˄ | 0.0000 | | 3a || 123.0 | 120.8 | -2%˄ || 8.13 | 8.27 | +2%˄ | 0.0000 | | 3b || 12.7 | 12.4 | -3%˄ || 78.67 | 80.74 | +3%˄ | 0.0000 | | 3c || 438.3 | 445.7 | +2%˄ || 2.28 | 2.24 | -2%˄ | 0.0000 | +| 4a || 113.5 | 101.9 | -10%˄ || 8.81 | 9.82 | +11%˄ | 0.0000 | | 4b || 11.5 | 11.3 | -2%˄ || 87.12 | 88.78 | +2%˄ | 0.0000 | +| 4c || 124.3 | 118.1 | -5%˄ || 8.04 | 8.47 | +5%˄ | 0.0000 | | 5a || 52.8 | 50.9 | -4%˄ || 18.93 | 19.65 | +4%˄ | 0.0000 | +| 5b || 135.0 | 120.5 | -11%˄ || 7.41 | 8.30 | +12%˄ | 0.0000 | | 5c || 183.2 | 188.2 | +3%˄ || 5.46 | 5.31 | -3%˄ | 0.0000 | | 6a || 103.9 | 102.5 | -1%˄ || 9.63 | 9.75 | +1%˄ | 0.0000 | | 6b || 115.9 | 114.5 | -1%˄ || 8.63 | 8.73 | +1%˄ | 0.0000 | | 6c || 98.7 | 97.7 | -1%˄ || 10.13 | 10.24 | +1%˄ | 0.0000 | | 6d || 531.1 | 527.6 | -1%˄ || 1.88 | 1.90 | +1%˄ | 0.0000 | | 6e || 103.6 | 102.3 | -1%˄ || 9.65 | 9.78 | +1%˄ | 0.0000 | | 6f || 703.0 | 697.5 | -1% || 1.42 | 1.43 | +1% | 0.0000 | | 7a || 53.3 | 52.9 | -1%˄ || 18.77 | 18.89 | +1%˄ | 0.0000 | | 7b || 49.7 | 49.7 | -0%˄ || 20.13 | 20.13 | +0%˄ | 0.9708 | | 7c || 591.2 | 571.2 | -3%˄ || 1.69 | 1.75 | +4%˄ | 0.0000 | | 8a || 38.2 | 37.3 | -2%˄ || 26.16 | 26.79 | +2%˄ | 0.0000 | | 8b || 35.8 | 35.4 | -1%˄ || 27.95 | 28.28 | +1%˄ | 0.0000 | | 8c || 1873.7 | 1907.9 | +2% || 0.53 | 0.52 | -2% | 0.0000 | -| 8d || 306.2 | 324.9 | +6%˄ || 3.27 | 3.08 | -6%˄ | 0.0000 | | 9a || 187.5 | 184.8 | -1%˄ || 5.33 | 5.41 | +1%˄ | 0.0000 | | 9b || 105.1 | 102.0 | -3%˄ || 9.52 | 9.80 | +3%˄ | 0.0000 | | 9c || 270.1 | 265.5 | -2%˄ || 3.70 | 3.77 | +2%˄ | 0.0000 | | 9d || 476.1 | 472.1 | -1%˄ || 2.10 | 2.12 | +1%˄ | 0.0000 | +---------++----------+---------+--------++----------+----------+--------+---------+ | Sum || 35082.5 | 35071.4 | -0% || | | | | | Geomean || | | || | | +1% | | +---------++----------+---------+--------++----------+----------+--------+---------+ | Notes || ˄ Execution stopped due to max runs reached | +---------++----------+---------+--------++----------+----------+--------+---------+ ```

hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 28 clients, 28 cores

Sum of avg. item runtimes: +5% || Geometric mean of throughput changes: -5%

Configuration Overview - click to expand

```diff +Configuration Overview---------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_1c24c5f49719e589324037c5037866a7388a034c_mt.json | /home/Marcel.Weisgut/hyrise/build-rel-clang-14/benchmark_all_results/hyriseBenchmarkJoinOrder_5acb2f2a0dac56837bcbe703f0e641a994c0fbb8_mt.json | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ | GIT-HASH | 1c24c5f49719e589324037c5037866a7388a034c-dirty | 5acb2f2a0dac56837bcbe703f0e641a994c0fbb8-dirty | | benchmark_mode | Shuffled | Shuffled | | build_type | release | release | | chunk_size | 65535 | 65535 | | clients | 28 | 28 | | compiler | clang 14.0.0 | clang 14.0.0 | | cores | 28 | 28 | | data_preparation_cores | 0 | 0 | | date | 2022-07-26 09:12:50 | 2022-07-25 23:40:39 | | encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} | | indexes | False | False | | max_duration | 1200000000000 | 1200000000000 | | max_runs | -1 | -1 | | time_unit | ns | ns | | using_scheduler | True | True | | utilized_cores_per_numa_node | [0, 28] | [0, 28] | | verify | False | False | | warmup_duration | 0 | 0 | +-------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------+ ```

```diff +---------++----------+---------+--------++----------+----------+--------+---------+ | Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value | | || old | new | || old | new | | | +---------++----------+---------+--------++----------+----------+--------+---------+ +| 10a || 198.5 | 147.1 | -26% || 0.51 | 0.49 | -5% | 0.0046 | -| 10b || 123.3 | 172.5 | +40% || 0.51 | 0.49 | -4% | 0.0567 | -| 10c || 401.6 | 420.2 | +5% || 0.51 | 0.49 | -5% | 0.5442 | +| 11a || 49.0 | 44.5 | -9% || 0.51 | 0.49 | -5% | 0.6760 | -| 11b || 38.1 | 48.2 | +27% || 0.51 | 0.49 | -5% | 0.3729 | +| 11c || 101.2 | 75.2 | -26% || 0.51 | 0.49 | -5% | 0.2850 | +| 11d || 72.2 | 66.9 | -7% || 0.51 | 0.49 | -5% | 0.6622 | -| 12a || 146.0 | 187.0 | +28% || 0.51 | 0.49 | -5% | 0.0517 | +| 12b || 80.2 | 57.3 | -29% || 0.51 | 0.49 | -5% | 0.2145 | -| 12c || 313.5 | 346.8 | +11% || 0.51 | 0.49 | -5% | 0.3445 | -| 13a || 590.9 | 637.6 | +8% || 0.51 | 0.49 | -5% | 0.2444 | -| 13b || 263.5 | 254.9 | -3% || 0.51 | 0.49 | -5% | 0.7370 | -| 13c || 238.4 | 236.9 | -1% || 0.51 | 0.49 | -5% | 0.9616 | -| 13d || 971.2 | 1008.9 | +4% || 0.51 | 0.49 | -5% | 0.4721 | -| 14a || 763.4 | 784.3 | +3% || 0.51 | 0.49 | -5% | 0.6430 | -| 14b || 432.1 | 490.3 | +13% || 0.51 | 0.49 | -5% | 0.0830 | -| 14c || 928.0 | 973.9 | +5% || 0.51 | 0.49 | -4% | 0.3864 | -| 15a || 164.8 | 169.8 | +3% || 0.51 | 0.49 | -5% | 0.8281 | +| 15b || 163.2 | 130.2 | -20% || 0.51 | 0.49 | -5% | 0.1816 | -| 15c || 155.3 | 167.3 | +8% || 0.51 | 0.49 | -5% | 0.6156 | -| 15d || 182.0 | 216.2 | +19% || 0.51 | 0.49 | -5% | 0.2427 | -| 16a || 2540.7 | 2559.9 | +1% || 0.51 | 0.49 | -5% | 0.7491 | -| 16b || 3719.8 | 3915.6 | +5% || 0.51 | 0.49 | -5% | 0.0014 | -| 16c || 2679.6 | 2847.7 | +6% || 0.51 | 0.49 | -4% | 0.0149 | -| 16d || 2581.4 | 2770.8 | +7% || 0.51 | 0.49 | -5% | 0.0024 | -| 17a || 785.0 | 830.6 | +6% || 0.51 | 0.49 | -5% | 0.3698 | -| 17b || 626.7 | 723.7 | +15% || 0.51 | 0.49 | -5% | 0.0656 | -| 17c || 546.4 | 577.0 | +6% || 0.51 | 0.49 | -5% | 0.5201 | +| 17d || 620.7 | 591.7 | -5% || 0.51 | 0.49 | -5% | 0.5292 | -| 17e || 1437.5 | 1597.8 | +11% || 0.51 | 0.49 | -5% | 0.0025 | -| 17f || 1012.8 | 1087.8 | +7% || 0.51 | 0.49 | -5% | 0.1316 | -| 18a || 712.1 | 815.2 | +14% || 0.51 | 0.49 | -5% | 0.0374 | +| 18b || 150.3 | 131.5 | -13% || 0.51 | 0.49 | -5% | 0.2111 | -| 18c || 607.0 | 694.3 | +14% || 0.51 | 0.49 | -5% | 0.0245 | -| 19a || 400.0 | 464.6 | +16% || 0.51 | 0.49 | -5% | 0.1033 | -| 19b || 326.4 | 333.2 | +2% || 0.51 | 0.49 | -5% | 0.8440 | -| 19c || 450.2 | 473.2 | +5% || 0.51 | 0.49 | -5% | 0.5601 | -| 19d || 1254.9 | 1223.8 | -2% || 0.51 | 0.49 | -5% | 0.5764 | -| 1a || 31.1 | 46.8 | +50% || 0.51 | 0.49 | -5% | 0.1967 | -| 1b || 29.5 | 38.2 | +29% || 0.51 | 0.49 | -5% | 0.4238 | -| 1c || 35.7 | 45.1 | +26% || 0.51 | 0.49 | -5% | 0.3646 | +| 1d || 29.1 | 26.6 | -9% || 0.51 | 0.49 | -5% | 0.6616 | -| 20a || 586.2 | 586.5 | +0% || 0.51 | 0.49 | -5% | 0.9951 | -| 20b || 427.7 | 417.5 | -2% || 0.51 | 0.49 | -5% | 0.7953 | -| 20c || 383.6 | 380.0 | -1% || 0.51 | 0.49 | -5% | 0.9088 | -| 21a || 68.2 | 78.9 | +16% || 0.51 | 0.49 | -5% | 0.4104 | -| 21b || 59.3 | 68.1 | +15% || 0.51 | 0.49 | -5% | 0.3970 | -| 21c || 65.4 | 66.4 | +1% || 0.51 | 0.49 | -5% | 0.9407 | -| 22a || 473.8 | 546.2 | +15% || 0.51 | 0.49 | -5% | 0.0642 | -| 22b || 345.7 | 353.1 | +2% || 0.51 | 0.49 | -5% | 0.8391 | -| 22c || 861.8 | 941.3 | +9% || 0.51 | 0.49 | -5% | 0.1590 | -| 22d || 1152.0 | 1269.3 | +10% || 0.51 | 0.49 | -5% | 0.0276 | -| 23a || 164.1 | 195.5 | +19% || 0.51 | 0.49 | -5% | 0.2546 | -| 23b || 109.9 | 115.0 | +5% || 0.51 | 0.49 | -5% | 0.7512 | +| 23c || 187.0 | 154.9 | -17% || 0.51 | 0.49 | -5% | 0.2233 | -| 24a || 252.1 | 256.7 | +2% || 0.51 | 0.49 | -5% | 0.8859 | -| 24b || 155.8 | 164.4 | +6% || 0.51 | 0.49 | -5% | 0.7391 | -| 25a || 440.7 | 545.0 | +24% || 0.51 | 0.49 | -5% | 0.0096 | -| 25b || 123.7 | 130.4 | +5% || 0.51 | 0.49 | -5% | 0.7204 | -| 25c || 937.7 | 1015.3 | +8% || 0.51 | 0.49 | -5% | 0.1508 | +| 26a || 356.4 | 301.9 | -15% || 0.51 | 0.49 | -5% | 0.1346 | -| 26b || 182.2 | 214.0 | +17% || 0.51 | 0.49 | -5% | 0.2805 | -| 26c || 407.1 | 490.5 | +20% || 0.51 | 0.49 | -4% | 0.0284 | -| 27a || 92.4 | 93.5 | +1% || 0.51 | 0.49 | -5% | 0.9570 | +| 27b || 120.4 | 87.1 | -28% || 0.51 | 0.49 | -5% | 0.1365 | -| 27c || 76.0 | 91.2 | +20% || 0.51 | 0.49 | -5% | 0.3630 | -| 28a || 620.4 | 749.5 | +21% || 0.51 | 0.49 | -5% | 0.0029 | +| 28b || 201.1 | 171.1 | -15% || 0.51 | 0.49 | -5% | 0.2667 | -| 28c || 749.7 | 732.9 | -2% || 0.51 | 0.49 | -5% | 0.7314 | +| 29a || 214.5 | 192.2 | -10% || 0.51 | 0.49 | -5% | 0.3261 | +| 29b || 137.2 | 124.3 | -9% || 0.51 | 0.49 | -5% | 0.5372 | -| 29c || 209.3 | 222.5 | +6% || 0.51 | 0.49 | -5% | 0.5668 | -| 2a || 190.7 | 186.0 | -2% || 0.51 | 0.49 | -5% | 0.8877 | -| 2b || 146.2 | 142.3 | -3% || 0.51 | 0.49 | -5% | 0.8640 | -| 2c || 57.7 | 97.4 | +69% || 0.51 | 0.49 | -5% | 0.0210 | +| 2d || 230.5 | 188.9 | -18% || 0.51 | 0.49 | -5% | 0.1602 | -| 30a || 301.1 | 293.7 | -2% || 0.51 | 0.49 | -5% | 0.7932 | +| 30b || 262.6 | 230.5 | -12% || 0.51 | 0.49 | -5% | 0.2974 | -| 30c || 743.6 | 802.4 | +8% || 0.51 | 0.49 | -5% | 0.3438 | -| 31a || 265.5 | 287.5 | +8% || 0.51 | 0.49 | -5% | 0.4643 | +| 31b || 163.6 | 150.6 | -8% || 0.51 | 0.49 | -5% | 0.5550 | +| 31c || 331.2 | 306.0 | -8% || 0.51 | 0.49 | -5% | 0.4593 | +| 32a || 45.7 | 37.7 | -18% || 0.51 | 0.49 | -5% | 0.4973 | -| 32b || 109.1 | 139.4 | +28% || 0.51 | 0.49 | -5% | 0.0848 | -| 33a || 62.5 | 80.3 | +29% || 0.51 | 0.49 | -5% | 0.2500 | -| 33b || 48.3 | 71.1 | +47% || 0.51 | 0.49 | -5% | 0.0637 | -| 33c || 69.3 | 76.3 | +10% || 0.51 | 0.49 | -5% | 0.6235 | -| 3a || 411.3 | 431.1 | +5% || 0.51 | 0.49 | -5% | 0.7009 | +| 3b || 67.6 | 53.6 | -21% || 0.51 | 0.49 | -5% | 0.3919 | -| 3c || 520.7 | 507.8 | -2% || 0.51 | 0.49 | -5% | 0.7445 | +| 4a || 189.0 | 179.1 | -5% || 0.51 | 0.49 | -5% | 0.6927 | -| 4b || 49.0 | 52.1 | +6% || 0.51 | 0.49 | -5% | 0.7967 | -| 4c || 236.9 | 229.5 | -3% || 0.51 | 0.49 | -5% | 0.8087 | -| 5a || 158.0 | 190.3 | +20% || 0.51 | 0.49 | -5% | 0.2473 | -| 5b || 174.7 | 200.5 | +15% || 0.51 | 0.49 | -5% | 0.2529 | +| 5c || 233.2 | 218.6 | -6% || 0.51 | 0.49 | -5% | 0.5864 | -| 6a || 91.3 | 111.4 | +22% || 0.51 | 0.49 | -5% | 0.1551 | -| 6b || 140.2 | 146.3 | +4% || 0.51 | 0.49 | -5% | 0.7520 | -| 6c || 91.7 | 91.4 | -0% || 0.51 | 0.49 | -5% | 0.9826 | -| 6d || 530.5 | 508.6 | -4% || 0.51 | 0.49 | -5% | 0.5292 | -| 6e || 89.8 | 124.6 | +39% || 0.51 | 0.49 | -5% | 0.0633 | -| 6f || 927.6 | 923.0 | -1% || 0.51 | 0.49 | -5% | 0.9072 | -| 7a || 115.3 | 131.2 | +14% || 0.51 | 0.49 | -5% | 0.4771 | +| 7b || 106.7 | 90.2 | -15% || 0.51 | 0.49 | -5% | 0.3308 | -| 7c || 801.6 | 930.5 | +16% || 0.51 | 0.49 | -5% | 0.0078 | -| 8a || 115.8 | 144.2 | +25% || 0.51 | 0.49 | -5% | 0.2055 | +| 8b || 156.6 | 142.4 | -9% || 0.51 | 0.49 | -5% | 0.6574 | -| 8c || 2166.8 | 2309.0 | +7% || 0.51 | 0.49 | -4% | 0.0141 | -| 8d || 641.5 | 628.7 | -2% || 0.51 | 0.49 | -5% | 0.7828 | -| 9a || 527.4 | 509.8 | -3% || 0.51 | 0.49 | -5% | 0.7477 | -| 9b || 321.8 | 391.8 | +22% || 0.51 | 0.49 | -5% | 0.0874 | -| 9c || 596.1 | 695.4 | +17% || 0.51 | 0.49 | -5% | 0.0531 | +| 9d || 948.9 | 871.2 | -8% || 0.51 | 0.49 | -5% | 0.0879 | +---------++----------+---------+--------++----------+----------+--------+---------+ -| Sum || 50552.4 | 53088.4 | +5% || | | | | -| Geomean || | | || | | -5% | | +---------++----------+---------+--------++----------+----------+--------+---------+ ```