StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.26k stars 1.67k forks source link

[Enhancement] No need to cache the file stats when enable_delta_column_statistics is false #47635

Open trueeyu opened 3 days ago

trueeyu commented 3 days ago

Why I'm doing:

What I'm doing:

No need to cache the file stats when enable_delta_column_statistics is false.

The purpose of doing this is to optimize memory usage, when there is so many files.

What type of PR is this:

Does this PR entail a change in behavior?

If yes, please specify the type of change:

Checklist:

Bugfix cherry-pick branch check:

github-actions[bot] commented 4 hours ago

[FE Incremental Coverage Report]

:white_check_mark: pass : 56 / 60 (93.33%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/connector/delta/DeltaLakeStats.java 0 3 00.00% [34, 35, 36]
:large_blue_circle: com/starrocks/connector/delta/ScanFileUtils.java 13 14 92.86% [64]
:large_blue_circle: com/starrocks/connector/delta/DeltaLakeMetadata.java 13 13 100.00% []
:large_blue_circle: com/starrocks/connector/delta/FileScanTask.java 17 17 100.00% []
:large_blue_circle: com/starrocks/connector/delta/DeltaUtils.java 1 1 100.00% []
:large_blue_circle: com/starrocks/connector/delta/DeltaStatisticProvider.java 6 6 100.00% []
:large_blue_circle: com/starrocks/planner/DeltaLakeScanNode.java 4 4 100.00% []
:large_blue_circle: com/starrocks/connector/delta/DeltaLakeMetadataFactory.java 2 2 100.00% []
github-actions[bot] commented 4 hours ago

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

sonarcloud[bot] commented 20 minutes ago

Quality Gate Passed Quality Gate passed

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud