search

vmware-archive / database-stream-processor

Streaming and Incremental Computation Framework
Other
225 stars 20 forks source link

Integration tests using a docker container #313

Closed lalithsuresh closed 1 year ago

lalithsuresh commented 1 year ago

Addresses #312

This patch builds and runs the docker container for the pipeline manager on every push. It then invokes script from another container to run tests against the pipeline manager container. We use two containers because container-container networking is easier on Github Actions than host-container networking.

As of now, the integration-tests.sh file only does a rudimentary curl call to see if a basic POST to /projects succeeds.

github-actions[bot] commented 1 year ago

Benchmark results

Nexmark

name main~6 [kOp/s] PR [kOp/s] Tput change [%] Assessment Peak RSS diff
q0 4019.52 3854.96 -4 :heavy_check_mark: -34.9 MB
q1 3985.7 3847.47 -3 :heavy_check_mark: 8.2 MB
q2 4023.28 3867.32 -4 :heavy_check_mark: -8.0 MB
q3 3300.74 3843.01 16 :evergreen_tree: 37.1 MB
q4 2977.89 2914.2 -2 :heavy_check_mark: -238.0 MB
q5 3961.28 3807.37 -4 :heavy_check_mark: -238.0 MB
q6 3180.54 3098.11 -3 :heavy_check_mark: -52.0 MB
q7 3186.33 2375.93 -25 :interrobang: 3.8 GB
q8 3900.39 3748.35 -4 :heavy_check_mark: 3.8 GB
q9 614.132 602.935 -2 :heavy_check_mark: 288.5 MB
q12 3788.28 3611.84 -5 :heavy_check_mark: 288.5 MB
q13 3242.08 2688.2 -17 :small_red_triangle_down: 288.5 MB
q14 4044.62 3877.47 -4 :heavy_check_mark: 288.5 MB
q15 4047.32 3882.79 -4 :heavy_check_mark: 288.5 MB
q16 1034.72 1032.83 0 :heavy_check_mark: 288.5 MB
q17 3049.53 3079.25 1 :heavy_check_mark: 288.5 MB
q18 888.182 890.187 0 :heavy_check_mark: -92.8 MB
q19 975.972 1001.79 3 :heavy_check_mark: -92.8 MB
q20 1100.14 1033.96 -6 :small_red_triangle_down: -92.8 MB
q21 3983.34 3785 -5 :heavy_check_mark: -92.8 MB
q22 4009.27 3805.22 -5 :small_red_triangle_down: -92.8 MB

Galen

name main~6 [s] PR [s] Runtime change [%] Assessment
galen 28.7097 28.5273 -1 :heavy_check_mark:

LDBC

algorithm dataset threads main~6 [kEVPS] PR [kEVPS] Tput change [%] Assessment Peak RSS diff
bfs graph500-22 1 1713.69 1920.62 12 :evergreen_tree: -20.5 kB
bfs datagen-8_4-fb 6 7567.17 8309.84 10 :evergreen_tree: -47.1 MB
pagerank graph500-22 1 682.774 685.862 0 :heavy_check_mark: 122.9 kB
pagerank datagen-8_4-fb 6 2052.73 2074.1 1 :heavy_check_mark: 373.1 MB

Nexmark (with Persistence)

name main~6 [kOp/s] PR [kOp/s] Tput change [%] PR DRAM [kOp/s] DRAM diff [%] Assessment
q0 2415.83 2438.96 1 2353.72 4 :heavy_check_mark:
q1 1719.34 1730.15 1 1682.17 3 :heavy_check_mark:
q2 2438.47 2393.83 -2 2376.81 1 :heavy_check_mark:
q3 2089.79 2098.06 0 2252.19 -7 :heavy_check_mark:
q4 441.731 434.883 -2 1405.21 -69 :heavy_check_mark:
q5 2093.61 2072.28 -1 2276.54 -9 :heavy_check_mark:
q6 412.037 400.545 -3 1384.02 -71 :heavy_check_mark:
q7 721.416 712.328 -1 1266.09 -44 :heavy_check_mark:
q8 2220 2227.37 0 2219.19 0 :heavy_check_mark:
q9 56.6337 58.4745 3 392.382 -85 :heavy_check_mark:
q12 911.264 881.491 -3 1773.48 -50 :heavy_check_mark:
q13 505.937 509.978 1 998.868 -49 :heavy_check_mark:
q14 1685.4 1710.45 1 1690.93 1 :heavy_check_mark:
q15 201.365 201.178 0 1153.37 -83 :heavy_check_mark:
q16 28.1206 28.3631 1 287.583 -90 :heavy_check_mark:
q17 90.8062 90.3762 0 807.203 -89 :heavy_check_mark:
q18 131.973 131.48 0 800.668 -84 :heavy_check_mark:
q19 195.763 197.041 1 649.761 -70 :heavy_check_mark:
q20 603.535 593.685 -2 941.657 -37 :heavy_check_mark:
q21 1521.07 1513.46 -1 1520.76 0 :heavy_check_mark:
q22 2127.15 2117.05 0 2071.78 2 :heavy_check_mark: