feat: request mode in raw SQL

[aceforeverd]

request mode in RAW SQL

SELECT ....
CONFIG ( execute_mode = 'request', values = (expr1, expr2 , ...) )

SELECT ....
CONFIG ( execute_mode = 'request', values = [(expr1, expr2 , ...), (....)] )

call procedure

call dp (expr1, expr2, ...)

TODOs

• docs
• support request value in system variable execute_mode
• take care const query

[github-actions[bot]]

SDK Test Report

102 files  ±0  102 suites  ±0   2m 22s :stopwatch: +9s 357 tests ±0  343 :white_check_mark: ±0  14 :zzz: ±0  0 :x: ±0  483 runs  ±0  469 :white_check_mark: ±0  14 :zzz: ±0  0 :x: ±0 

Results for commit 4c058269. ± Comparison against base commit d33e2c39.

This pull request removes 30 and adds 9 tests. Note that renamed tests count towards both.

``` PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1 PARTITION BY t1.col2 ORDER BY t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](1) ) limit 10;](2) ) limit 10;](3) FROM db1.t1 FROM t1 WINDOW w1 AS ( last join db2.t2 order by db2.t2.col1 … ``` ``` com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[, SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1 FROM db1.t1 last join db2.t2 order by db2.t2.col1 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0 WINDOW w1 AS ( PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](2) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1, SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0 WINDOW w1 AS ( PARTITION BY t1.col2 ORDER BY t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](1) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null, SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1 FROM db1.t1 last join db2.t2 order by db2.t2.col1 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0 WINDOW w1 AS ( PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](3) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Column Not found: db1.t2.str1](2) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Column Not found: .t2.col1](3) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Column Not found: .t2.str1](1) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1 FROM t1 last join t2 order by t2.col1 on t1.col1 = t2.col1 and t1.col2 = t2.col0 WINDOW w1 AS ( PARTITION BY t1.col2 ORDER BY t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](1) ```

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

HybridSE Mac Test Report

20 146 tests   20 144 :white_check_mark:  9m 15s :stopwatch:    256 suites       2 :zzz:     68 files         0 :x:

Results for commit 4c058269.

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

HybridSE Linux Test Report

20 146 tests   20 144 :white_check_mark:  6m 20s :stopwatch:    256 suites       2 :zzz:     68 files         0 :x:

Results for commit 4c058269.

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

Linux Test Report

    57 files  +     5     247 suites  +190   1h 45m 54s :stopwatch: + 1h 20m 27s 13 229 tests +12 675  13 222 :white_check_mark: +12 672  7 :zzz: +3  0 :x: ±0  18 776 runs  +18 221  18 769 :white_check_mark: +18 218  7 :zzz: +3  0 :x: ±0 

Results for commit 4c058269. ± Comparison against base commit d33e2c39.

:recycle: This comment has been updated with latest results.

[aceforeverd]

TODO:

• auto infer from CONFIG values data type

[aceforeverd]

execute_mode rules

Values

execute_mode in SQL CONFIG clause can be

• online
• request
• offline
• batchrequest (hidden, may removed in later release when 'request' feature complete. Users are generally aware about the three options above )

values option in SQL CONFIG clause can be

• StructConstructorWithParens ( e.g ( expr1, expr2, ...) )
• Array [ StructConstructorWithParens ] ( e.g [ (expr1, expr2, ...), ( ...), ... ] )

execute_mode in system variable can be

• online
• offline
• request ( TODO later, won't support in this PR)

Rules

• determine execute_mode
  • take execute_mode in CONFIG clause IF EXISTS
  • OTHERWISE take execute_mode from system variable execute_mode (pass in from sql_cluster_router client)
• execute_mode to engine_mode in SQL compiler
  • online -> batch
  • offline -> offline
  • request ->
    • values defines single row: request
    • values defines multiple row: batchrequest
  • batchrequest -> batchrequest
• Special cases
  • engine_mode = 'request' | 'batchrequest' and request table not empty
  • COMPILE ERROR: when CONFIG values does not match expected schema of request table
  • COMPILE ERROR: when CONFIG values is empty array
  • engine_mode = 'request' | 'batchrequest' and request table is empty (a const query without table)
  • CONFIG values can be empty struct or empty array.
  • This PR won't take care of this case, TODO later
• CONFIG values ignored if engine_mode is batch or offline