StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.29k stars 1.68k forks source link

[Enhancement] Optimize the mem usage of collect stats for delta lake #47817

Closed trueeyu closed 2 days ago

trueeyu commented 2 days ago

Why I'm doing:

What I'm doing:

No need to cache the stats in ScanFileTask, just drop it after adding up.

What type of PR is this:

Does this PR entail a change in behavior?

If yes, please specify the type of change:

Checklist:

Bugfix cherry-pick branch check:

sonarcloud[bot] commented 2 days ago

Quality Gate Passed Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions[bot] commented 2 days ago

[FE Incremental Coverage Report]

:white_check_mark: pass : 32 / 32 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/connector/delta/ScanFileUtils.java 3 3 100.00% []
:large_blue_circle: com/starrocks/connector/delta/DeltaLakeFileStats.java 6 6 100.00% []
:large_blue_circle: com/starrocks/connector/delta/DeltaLakeMetadata.java 14 14 100.00% []
:large_blue_circle: com/starrocks/connector/delta/DeltaStatisticProvider.java 9 9 100.00% []
github-actions[bot] commented 2 days ago

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)