vmware-archive / database-stream-processor

Streaming and Incremental Computation Framework
Other
222 stars 20 forks source link

Web UI initial import #358

Closed gz closed 1 year ago

gz commented 1 year ago

TODO before check-in:

codecov[bot] commented 1 year ago

Codecov Report

Merging #358 (5cb1b0c) into main (dbe4663) will increase coverage by 0.15%. The diff coverage is 70.09%.

:exclamation: Current head 5cb1b0c differs from pull request most recent head a2bbcae. Consider uploading reports for the commit a2bbcae to get more accurate results

Additional details and impacted files [![Impacted file tree graph](https://codecov.io/gh/vmware/database-stream-processor/pull/358/graphs/tree.svg?width=650&height=150&src=pr&token=0wZcmD11gt&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware)](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) ```diff @@ Coverage Diff @@ ## main #358 +/- ## ========================================== + Coverage 72.99% 73.14% +0.15% ========================================== Files 237 237 Lines 49568 50146 +578 ========================================== + Hits 36180 36680 +500 - Misses 13388 13466 +78 ``` | [Impacted Files](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) | Coverage Δ | | |---|---|---| | [crates/dataflow-jit/src/codegen/intrinsics.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi9pbnRyaW5zaWNzLnJz) | `61.60% <0.00%> (-8.93%)` | :arrow_down: | | [crates/dataflow-jit/src/dataflow/mod.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvZGF0YWZsb3cvbW9kLnJz) | `49.88% <0.00%> (-10.08%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/function/passes.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvZnVuY3Rpb24vcGFzc2VzLnJz) | `52.57% <ø> (ø)` | | | [crates/dataflow-jit/src/ir/ids.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvaWRzLnJz) | `61.29% <0.00%> (-2.05%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/nodes/aggregate.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvYWdncmVnYXRlLnJz) | `10.48% <ø> (-0.40%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/nodes/constant.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvY29uc3RhbnQucnM=) | `47.74% <ø> (+1.25%)` | :arrow_up: | | [crates/dataflow-jit/src/ir/nodes/differentiate.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvZGlmZmVyZW50aWF0ZS5ycw==) | `17.18% <ø> (+1.47%)` | :arrow_up: | | [crates/dataflow-jit/src/ir/nodes/filter\_map.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvZmlsdGVyX21hcC5ycw==) | `41.62% <ø> (+0.32%)` | :arrow_up: | | [crates/dataflow-jit/src/ir/nodes/flat\_map.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvZmxhdF9tYXAucnM=) | `81.13% <ø> (-1.02%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/nodes/index.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvaW5kZXgucnM=) | `57.57% <ø> (-1.85%)` | :arrow_down: | | ... and [41 more](https://codecov.io/gh/vmware/database-stream-processor/pull/358?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) | | ... and [3 files with indirect coverage changes](https://codecov.io/gh/vmware/database-stream-processor/pull/358/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware)
github-actions[bot] commented 1 year ago

Benchmark results

Nexmark

name main~26 [kOp/s] PR [kOp/s] Tput change [%] Assessment Peak RSS diff
q0 5262.63 4266.87 -19 :small_red_triangle_down: 20.1 MB
q1 5429.6 5457.16 1 :heavy_check_mark: 12.1 MB
q2 5361.6 5358.9 0 :heavy_check_mark: 4.4 MB
q3 5321.94 5455.31 3 :heavy_check_mark: 47.9 MB
q4 4802.55 4453.2 -7 :small_red_triangle_down: -380.6 MB
q5 5378.33 5372.31 0 :heavy_check_mark: -380.6 MB
q6 4880.13 4850.02 -1 :heavy_check_mark: -314.7 MB
q7 2559.31 4194.39 64 :evergreen_tree: -6.1 GB
q8 5161.2 5088.75 -1 :heavy_check_mark: -6.1 GB
q9 875.448 866.817 -1 :heavy_check_mark: 482.7 MB
q12 4910.28 5046.03 3 :heavy_check_mark: 482.7 MB
q13 3645.17 3560.34 -2 :heavy_check_mark: 482.7 MB
q14 5481.43 5509.23 1 :heavy_check_mark: 482.7 MB
q15 4895.11 5325.83 9 :evergreen_tree: 482.7 MB
q16 1045.16 1040.05 0 :heavy_check_mark: 482.7 MB
q17 3134.56 3182.34 2 :heavy_check_mark: -284.8 MB
q18 1440.13 1466.79 2 :heavy_check_mark: 467.5 MB
q19 1435.15 1419.09 -1 :heavy_check_mark: 467.5 MB
q20 1500.83 1537.87 2 :heavy_check_mark: 467.5 MB
q21 5117.36 5220.91 2 :heavy_check_mark: 467.5 MB
q22 5356.59 5398.67 1 :heavy_check_mark: 467.5 MB

Galen

name main~26 [s] PR [s] Runtime change [%] Assessment
galen 28.6952 28.6648 0 :heavy_check_mark:

LDBC

algorithm dataset threads main~26 [kEVPS] PR [kEVPS] Tput change [%] Assessment Peak RSS diff
bfs graph500-22 1 1835.7 1775.24 -3 :heavy_check_mark: -86.0 kB
bfs datagen-8_4-fb 6 8173.61 7658.89 -6 :small_red_triangle_down: 41.9 MB
pagerank graph500-22 1 681.655 692.045 2 :heavy_check_mark: -118.8 kB
pagerank datagen-8_4-fb 6 1994.79 2030.99 2 :heavy_check_mark: -150.4 MB

Nexmark (with Persistence)

name main~26 [kOp/s] PR [kOp/s] Tput change [%] PR DRAM [kOp/s] DRAM diff [%] Assessment
q0 2399.23 2444.9 2 2399.47 2 :heavy_check_mark:
q1 1713.93 1709.85 0 1666.91 3 :heavy_check_mark:
q2 2407.67 2429.89 1 2336.92 4 :heavy_check_mark:
q3 2036.93 2073.46 2 2260.16 -8 :heavy_check_mark:
q4 360.256 362.387 1 1378.43 -74 :heavy_check_mark:
q5 2015.3 2030.62 1 2242.61 -9 :heavy_check_mark:
q6 332.421 339.702 2 1377.67 -75 :heavy_check_mark:
q7 639.584 665.907 4 1310.45 -49 :heavy_check_mark:
q8 2171.19 2283.95 5 2196.86 4 :evergreen_tree:
q9 79.0764 80.4758 2 385.606 -79 :heavy_check_mark:
q12 838.481 880.186 5 1790.59 -51 :heavy_check_mark:
q13 443.491 442.99 0 998.11 -56 :heavy_check_mark:
q14 1697.52 1682.13 -1 1614.97 4 :heavy_check_mark:
q15 198.112 198.22 0 1178.96 -83 :heavy_check_mark:
q16 25.5771 25.7467 1 285.279 -91 :heavy_check_mark:
q17 81.4521 81.5963 0 786.609 -90 :heavy_check_mark:
q18 124.763 125.325 0 784.663 -84 :heavy_check_mark:
q19 182.489 180.629 -1 649.054 -72 :heavy_check_mark:
q20 505.761 523.408 3 917.406 -43 :heavy_check_mark:
q21 1489.8 1435.29 -4 1484.57 -3 :heavy_check_mark:
q22 2102.89 2017.96 -4 2023.69 0 :heavy_check_mark:
gz commented 1 year ago

@mbudiu-vmw sorry for the huge PR

I suggest to focus reviewing to most relevant folders with core-logic: