search

vmware-archive / database-stream-processor

Streaming and Incremental Computation Framework
Other
225 stars 20 forks source link

Refactor runner code to support multiple runner types #396

Closed lalithsuresh closed 1 year ago

lalithsuresh commented 1 year ago

See individual commits

github-actions[bot] commented 1 year ago

Benchmark results

Nexmark

name main~68 [kOp/s] PR [kOp/s] Tput change [%] Assessment Peak RSS diff
q0 5262.63 5292.9 1 :heavy_check_mark: 13.3 MB
q1 5429.6 5341.68 -2 :heavy_check_mark: 19.4 MB
q2 5361.6 5308.36 -1 :heavy_check_mark: 11.2 MB
q3 5321.94 5307.67 0 :heavy_check_mark: 117.3 MB
q4 4802.55 4718.72 -2 :heavy_check_mark: -231.2 MB
q5 5378.33 5205.98 -3 :heavy_check_mark: -231.2 MB
q6 4880.13 4769.69 -2 :heavy_check_mark: -173.9 MB
q7 2559.31 2814.38 10 :evergreen_tree: -2.9 GB
q8 5161.2 5078.67 -2 :heavy_check_mark: -2.9 GB
q9 875.448 861.367 -2 :heavy_check_mark: 174.8 MB
q12 4910.28 5003.27 2 :heavy_check_mark: 174.8 MB
q13 3645.17 2786.44 -24 :small_red_triangle_down: 174.8 MB
q14 5481.43 4233.47 -23 :small_red_triangle_down: 174.8 MB
q15 4895.11 5230.8 7 :evergreen_tree: 174.8 MB
q16 1045.16 1036.34 -1 :heavy_check_mark: 174.8 MB
q17 3134.56 3236.78 3 :heavy_check_mark: -199.7 MB
q18 1440.13 1445.56 0 :heavy_check_mark: 764.3 MB
q19 1435.15 1442.24 0 :heavy_check_mark: 764.3 MB
q20 1500.83 1534.04 2 :heavy_check_mark: 764.3 MB
q21 5117.36 5064.83 -1 :heavy_check_mark: 764.3 MB
q22 5356.59 5220.41 -3 :heavy_check_mark: 764.3 MB

Galen

name main~68 [s] PR [s] Runtime change [%] Assessment
galen 28.6952 29.2037 2 :heavy_check_mark:

LDBC

algorithm dataset threads main~68 [kEVPS] PR [kEVPS] Tput change [%] Assessment Peak RSS diff
bfs graph500-22 1 1835.7 1761.79 -4 :heavy_check_mark: -131.1 kB
bfs datagen-8_4-fb 6 8173.61 7729.57 -5 :small_red_triangle_down: 32.4 MB
pagerank graph500-22 1 681.655 683.323 0 :heavy_check_mark: 16.4 kB
pagerank datagen-8_4-fb 6 1994.79 2035.62 2 :heavy_check_mark: -153.9 MB

Nexmark (with Persistence)

name main~68 [kOp/s] PR [kOp/s] Tput change [%] PR DRAM [kOp/s] DRAM diff [%] Assessment
q0 2399.23 2434.22 1 2390.35 2 :heavy_check_mark:
q1 1713.93 1711.02 0 1693.44 1 :heavy_check_mark:
q2 2407.67 2438.24 1 2333.5 4 :heavy_check_mark:
q3 2036.93 2066.94 1 2236.49 -8 :heavy_check_mark:
q4 360.256 366.64 2 1396.12 -74 :heavy_check_mark:
q5 2015.3 2060.82 2 2238.44 -8 :heavy_check_mark:
q6 332.421 342.138 3 1391.99 -75 :heavy_check_mark:
q7 639.584 642.519 0 1278.44 -50 :heavy_check_mark:
q8 2171.19 2300.26 6 2209.85 4 :evergreen_tree:
q9 79.0764 79.5933 1 386.829 -79 :heavy_check_mark:
q12 838.481 879.451 5 1777.49 -51 :heavy_check_mark:
q13 443.491 445.565 0 993.268 -55 :heavy_check_mark:
q14 1697.52 1705.33 0 1686.04 1 :heavy_check_mark:
q15 198.112 198.259 0 1183.79 -83 :heavy_check_mark:
q16 25.5771 25.4675 0 282.635 -91 :heavy_check_mark:
q17 81.4521 80.8969 -1 805.127 -90 :heavy_check_mark:
q18 124.763 123.808 -1 800.233 -85 :heavy_check_mark:
q19 182.489 182.051 0 644.82 -72 :heavy_check_mark:
q20 505.761 526.394 4 934.403 -44 :heavy_check_mark:
q21 1489.8 1529.68 3 1518.76 1 :heavy_check_mark:
q22 2102.89 2106.26 0 2048.71 3 :heavy_check_mark:
codecov[bot] commented 1 year ago

Codecov Report

Merging #396 (0ef84da) into main (dbe4663) will increase coverage by 1.15%. The diff coverage is 68.08%.

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396/graphs/tree.svg?width=650&height=150&src=pr&token=0wZcmD11gt&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware)](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) ```diff @@ Coverage Diff @@ ## main #396 +/- ## ========================================== + Coverage 72.99% 74.14% +1.15% ========================================== Files 237 237 Lines 49568 52331 +2763 ========================================== + Hits 36180 38800 +2620 - Misses 13388 13531 +143 ``` | [Impacted Files](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) | Coverage Δ | | |---|---|---| | [crates/dataflow-jit/src/codegen/vtable/cmp.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi92dGFibGUvY21wLnJz) | `100.00% <ø> (ø)` | | | [crates/dataflow-jit/src/codegen/vtable/debug.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi92dGFibGUvZGVidWcucnM=) | `100.00% <ø> (ø)` | | | [crates/dataflow-jit/src/codegen/vtable/hash.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi92dGFibGUvaGFzaC5ycw==) | `100.00% <ø> (ø)` | | | [crates/dataflow-jit/src/codegen/vtable/mod.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi92dGFibGUvbW9kLnJz) | `92.17% <ø> (-1.45%)` | :arrow_down: | | [crates/dataflow-jit/src/facade.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvZmFjYWRlLnJz) | `86.06% <ø> (+86.06%)` | :arrow_up: | | [crates/dataflow-jit/src/ir/exprs/call.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvZXhwcnMvY2FsbC5ycw==) | `66.00% <0.00%> (-4.22%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/function/passes.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvZnVuY3Rpb24vcGFzc2VzLnJz) | `55.28% <ø> (+2.70%)` | :arrow_up: | | [crates/dataflow-jit/src/ir/nodes/aggregate.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvYWdncmVnYXRlLnJz) | `10.48% <ø> (-0.40%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/nodes/constant.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvY29uc3RhbnQucnM=) | `63.06% <ø> (+16.57%)` | :arrow_up: | | [crates/dataflow-jit/src/ir/nodes/differentiate.rs](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvZGlmZmVyZW50aWF0ZS5ycw==) | `17.18% <ø> (+1.47%)` | :arrow_up: | | ... and [78 more](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) | | ... and [11 files with indirect coverage changes](https://app.codecov.io/gh/vmware/database-stream-processor/pull/396/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware)