4paradigm / OpenMLDB

OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.
https://openmldb.ai
Apache License 2.0
1.59k stars 321 forks source link

fix: drop aggr tables in drop table #3908

Closed vagetablechicken closed 4 months ago

vagetablechicken commented 6 months ago

closes #3836

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 96.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 77.88%. Comparing base (1c1e213) to head (b7b87b2).

Files Patch % Lines
src/sdk/sql_cluster_router.cc 88.88% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #3908 +/- ## ============================================ + Coverage 74.78% 77.88% +3.10% ============================================ Files 762 566 -196 Lines 137523 125622 -11901 Branches 1535 0 -1535 ============================================ - Hits 102844 97846 -4998 + Misses 34376 27776 -6600 + Partials 303 0 -303 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 6 months ago

SDK Test Report

100 files   -  2  100 suites   - 2   2m 7s :stopwatch: -13s 318 tests  - 41  308 :white_check_mark:  - 37  10 :zzz:  - 4  0 :x: ±0  446 runs   - 41  436 :white_check_mark:  - 37  10 :zzz:  - 4  0 :x: ±0 

Results for commit b7b87b2d. ± Comparison against base commit 1c1e2134.

This pull request removes 71 and adds 9 tests. Note that renamed tests count towards both. ``` PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1 PARTITION BY t1.col2 ORDER BY t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](1) ) limit 10;](2) ) limit 10;](3) FROM db1.t1 FROM t1 WINDOW w1 AS ( last join db2.t2 order by db2.t2.col1 … ``` ``` com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[, SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1 FROM db1.t1 last join db2.t2 order by db2.t2.col1 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0 WINDOW w1 AS ( PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](2) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1, SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0 WINDOW w1 AS ( PARTITION BY t1.col2 ORDER BY t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](1) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null, SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1 FROM db1.t1 last join db2.t2 order by db2.t2.col1 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0 WINDOW w1 AS ( PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](3) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Column Not found: db1.t2.str1](2) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Column Not found: .t2.col1](3) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Column Not found: .t2.str1](1) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1 FROM t1 last join db2.t2 order by db2.t2.col1 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0; , SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5) com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1 FROM t1 last join t2 order by t2.col1 on t1.col1 = t2.col1 and t1.col2 = t2.col0 WINDOW w1 AS ( PARTITION BY t1.col2 ORDER BY t1.col1 ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW ) limit 10;](1) ```

:recycle: This comment has been updated with latest results.

github-actions[bot] commented 6 months ago

Linux Test Report

    60 files  ±0     253 suites  ±0   1h 58m 20s :stopwatch: + 18m 12s 13 605 tests ±0  13 598 :white_check_mark: ±0  7 :zzz: ±0  0 :x: ±0  19 317 runs  ±0  19 310 :white_check_mark: ±0  7 :zzz: ±0  0 :x: ±0 

Results for commit b7b87b2d. ± Comparison against base commit 1c1e2134.

:recycle: This comment has been updated with latest results.

vagetablechicken commented 6 months ago

truncate base table, will aggr table be cleaned?

vagetablechicken commented 6 months ago

truncate base table, will aggr table be cleaned?

https://github.com/4paradigm/OpenMLDB/blob/c2d9c8df8036418f2a0acb04c2a693f0715dac36/src/tablet/tablet_impl.cc#L3484-L3506

GetAggregators will get all aggr tables of base table, and truncate it.