Misc JIT improvements - Githubissues

Kixiron commented 1 year ago

Fixed #374 with more robust checks
Implemented the following functions:
- @dbsp.str.is_nfc()
- @dbsp.str.is_nfd()
- @dbsp.str.is_nfkc()
- @dbsp.str.is_nfkd()
- @dbsp.str.is_lowercase()
- @dbsp.str.is_uppercase()
- @dbsp.str.bit_length()
- @dbsp.str.byte_length()
- @dbsp.str.char_length()
Changed the JSON repr of Neg cc @mbudiu-vmw
- Now has the layout field instead of output_layout which contains a StreamLayout

codecov[bot] commented 1 year ago

Codecov Report

Merging #381 (fd01a45) into main (f463b48) will increase coverage by 0.90%. The diff coverage is 71.41%.

Additional details and impacted files

[![Impacted file tree graph](https://codecov.io/gh/vmware/database-stream-processor/pull/381/graphs/tree.svg?width=650&height=150&src=pr&token=0wZcmD11gt&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware)](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) ```diff @@ Coverage Diff @@ ## main #381 +/- ## ========================================== + Coverage 72.23% 73.14% +0.90% ========================================== Files 233 237 +4 Lines 48316 50171 +1855 ========================================== + Hits 34903 36698 +1795 - Misses 13413 13473 +60 ``` | [Impacted Files](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) | Coverage Δ | | |---|---|---| | [crates/dataflow-jit/src/codegen/intrinsics.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi9pbnRyaW5zaWNzLnJz) | `61.60% <0.00%> (-12.89%)` | :arrow_down: | | [crates/dataflow-jit/src/codegen/mod.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvY29kZWdlbi9tb2QucnM=) | `52.75% <0.00%> (-0.41%)` | :arrow_down: | | [crates/dataflow-jit/src/dataflow/nodes.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvZGF0YWZsb3cvbm9kZXMucnM=) | `57.14% <0.00%> (-1.69%)` | :arrow_down: | | [crates/dataflow-jit/src/facade.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvZmFjYWRlLnJz) | `0.00% <0.00%> (ø)` | | | [crates/dataflow-jit/src/ir/exprs/select.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvZXhwcnMvc2VsZWN0LnJz) | `61.53% <0.00%> (ø)` | | | [crates/dataflow-jit/src/ir/nodes/join.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvam9pbi5ycw==) | `41.30% <0.00%> (-1.19%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/nodes/subgraph.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvbm9kZXMvc3ViZ3JhcGgucnM=) | `61.46% <0.00%> (-1.04%)` | :arrow_down: | | [crates/dataflow-jit/src/ir/visit.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvaXIvdmlzaXQucnM=) | `17.54% <0.00%> (-0.43%)` | :arrow_down: | | [crates/dataflow-jit/src/lib.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2RhdGFmbG93LWppdC9zcmMvbGliLnJz) | `100.00% <ø> (ø)` | | | [crates/dbsp/src/monitor/mod.rs](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware#diff-Y3JhdGVzL2Ric3Avc3JjL21vbml0b3IvbW9kLnJz) | `71.29% <ø> (ø)` | | | ... and [67 more](https://codecov.io/gh/vmware/database-stream-processor/pull/381?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware) | | ... and [2 files with indirect coverage changes](https://codecov.io/gh/vmware/database-stream-processor/pull/381/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=vmware)

github-actions[bot] commented 1 year ago

Benchmark results

Nexmark

1 out of 21 queries have regressed :exclamation:
Compared results from dbe4663 (main) with fd01a45 (PR) No benchmark results found for current main revision, compared against dbe46638028c8dc4fbf95aab22bc94ff4fed9c24

name	main~3 [kOp/s]	PR [kOp/s]	Tput change [%]	Assessment	Peak RSS diff
q0	5262.63	5437.69	3	:heavy_check_mark:	6.2 MB
q1	5429.6	5460.96	1	:heavy_check_mark:	15.2 MB
q2	5361.6	5417.93	1	:heavy_check_mark:	13.6 MB
q3	5321.94	5418.23	2	:heavy_check_mark:	45.1 MB
q4	4802.55	4874.39	1	:heavy_check_mark:	-412.8 MB
q5	5378.33	4316.06	-20	:small_red_triangle_down:	-412.8 MB
q6	4880.13	4887.37	0	:heavy_check_mark:	-60.4 MB
q7	2559.31	3254.29	27	:evergreen_tree:	-3.7 GB
q8	5161.2	5114.21	-1	:heavy_check_mark:	-3.7 GB
q9	875.448	866.048	-1	:heavy_check_mark:	-112.4 MB
q12	4910.28	5049.73	3	:heavy_check_mark:	-112.4 MB
q13	3645.17	3544.79	-3	:heavy_check_mark:	-112.4 MB
q14	5481.43	5452.18	-1	:heavy_check_mark:	-112.4 MB
q15	4895.11	5412.94	11	:evergreen_tree:	-112.4 MB
q16	1045.16	1043.72	0	:heavy_check_mark:	-112.4 MB
q17	3134.56	3073.79	-2	:heavy_check_mark:	12.5 MB
q18	1440.13	1420.33	-1	:heavy_check_mark:	138.7 MB
q19	1435.15	1371.62	-4	:heavy_check_mark:	138.7 MB
q20	1500.83	1528.62	2	:heavy_check_mark:	138.7 MB
q21	5117.36	5331.96	4	:heavy_check_mark:	138.7 MB
q22	5356.59	5422.85	1	:heavy_check_mark:	138.7 MB

Galen

Compared results from dbe4663 (main) with fd01a45 (PR) No benchmark results found for current main revision, compared against dbe46638028c8dc4fbf95aab22bc94ff4fed9c24

name	main~3 [s]	PR [s]	Runtime change [%]	Assessment
galen	28.6952	29.6835	3	:heavy_check_mark:

LDBC

Compared results from dbe4663 (main) with fd01a45 (PR) No benchmark results found for current main revision, compared against dbe46638028c8dc4fbf95aab22bc94ff4fed9c24

algorithm	dataset	threads	main~3 [kEVPS]	PR [kEVPS]	Tput change [%]	Assessment	Peak RSS diff
bfs	graph500-22	1	1835.7	1772.23	-3	:heavy_check_mark:	204.8 kB
bfs	datagen-8_4-fb	6	8173.61	7818.94	-4	:heavy_check_mark:	417.0 MB
pagerank	graph500-22	1	681.655	688.875	1	:heavy_check_mark:	-81.9 kB
pagerank	datagen-8_4-fb	6	1994.79	2062.94	3	:heavy_check_mark:	-207.7 MB

Nexmark (with Persistence)

Compared results from dbe4663 (main) with fd01a45 (PR) No benchmark results found for current main revision, compared against dbe46638028c8dc4fbf95aab22bc94ff4fed9c24

name	main~3 [kOp/s]	PR [kOp/s]	Tput change [%]	PR DRAM [kOp/s]	DRAM diff [%]	Assessment
q0	2399.23	2451.48	2	2350.75	4	:heavy_check_mark:
q1	1713.93	1677.79	-2	1559.19	8	:heavy_check_mark:
q2	2407.67	2469.49	3	2326.85	6	:heavy_check_mark:
q3	2036.93	2057.59	1	2276.34	-10	:heavy_check_mark:
q4	360.256	358.85	0	1406.66	-74	:heavy_check_mark:
q5	2015.3	2042.69	1	2240.33	-9	:heavy_check_mark:
q6	332.421	338.587	2	1380.99	-75	:heavy_check_mark:
q7	639.584	662.686	4	1285.79	-48	:heavy_check_mark:
q8	2171.19	2271.62	5	2220.52	2	:heavy_check_mark:
q9	79.0764	79.4857	1	381.192	-79	:heavy_check_mark:
q12	838.481	849.255	1	1782.09	-52	:heavy_check_mark:
q13	443.491	443.713	0	981.391	-55	:heavy_check_mark:
q14	1697.52	1674.35	-1	1687.86	-1	:heavy_check_mark:
q15	198.112	198.943	0	1189.73	-83	:heavy_check_mark:
q16	25.5771	26.2544	3	284.983	-91	:heavy_check_mark:
q17	81.4521	81.4686	0	790.269	-90	:heavy_check_mark:
q18	124.763	126.035	1	785.505	-84	:heavy_check_mark:
q19	182.489	183.101	0	642.768	-72	:heavy_check_mark:
q20	505.761	510.263	1	866.405	-41	:heavy_check_mark:
q21	1489.8	1519.51	2	1493	2	:heavy_check_mark:
q22	2102.89	2029.03	-4	2055.72	-1	:heavy_check_mark:

Kixiron commented 1 year ago

@lalithsuresh Any idea what's going on with the build-and-push-image job? It's giving a curl: (6) Could not resolve host: dbsp error

ryzhyk commented 1 year ago

@lalithsuresh Any idea what's going on with the build-and-push-image job? It's giving a curl: (6) Could not resolve host: dbsp error

I believe he's already looking into it, and it's unrelated to this PR.

lalithsuresh commented 1 year ago

You can ignore it for now. It's from the migration to pg-embed in #375. I'm fixing it in a separate PR.

Kixiron commented 1 year ago

@ryzhyk the issue is that I can't merge since the job is failing

vmware-archive / database-stream-processor

Misc JIT improvements #381

Codecov Report

Benchmark results

Nexmark

Galen

LDBC

Nexmark (with Persistence)