StarRocks / starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.
https://starrocks.io
Apache License 2.0
9.04k stars 1.82k forks source link

[Enhancement] Opt memory tracker for FE #53055

Closed gengjun-git closed 18 minutes ago

gengjun-git commented 2 days ago

Why I'm doing:

SizeEstimator.estimate will calculate the retained size of the object, that is the size of the param object and all the referenced objects. This will lead to repeated counting when the same object is referenced by many objects, and result in very high resource usage when the param object refers to network object or GlobalMetaStore.

What I'm doing:

Change to ClassLayout to only calculate the shadow size of the object.

Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

If yes, please specify the type of change:

Checklist:

Bugfix cherry-pick branch check:

HangyuanLiu commented 4 hours ago

Will this code still work for other types of JDK?

gengjun-git commented 3 hours ago

@mergifyio rebase

mergify[bot] commented 3 hours ago

rebase

✅ Branch has been successfully rebased

sonarcloud[bot] commented 2 hours ago

Quality Gate Failed Quality Gate failed

Failed conditions
B Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

github-actions[bot] commented 1 hour ago

[Java-Extensions Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] commented 1 hour ago

[FE Incremental Coverage Report]

:white_check_mark: pass : 5 / 5 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
:large_blue_circle: com/starrocks/memory/MemoryTrackable.java 5 5 100.00% []
github-actions[bot] commented 1 hour ago

[BE Incremental Coverage Report]

:white_check_mark: pass : 0 / 0 (0%)

github-actions[bot] commented 18 minutes ago

@Mergifyio backport branch-3.4

github-actions[bot] commented 18 minutes ago

@Mergifyio backport branch-3.3

github-actions[bot] commented 18 minutes ago

@Mergifyio backport branch-3.2

mergify[bot] commented 18 minutes ago

backport branch-3.4

✅ Backports have been created

* [#53142 [Enhancement] Opt memory tracker for FE (backport #53055)](https://github.com/StarRocks/starrocks/pull/53142) has been created for branch `branch-3.4`
mergify[bot] commented 18 minutes ago

backport branch-3.3

✅ Backports have been created

* [#53143 [Enhancement] Opt memory tracker for FE (backport #53055)](https://github.com/StarRocks/starrocks/pull/53143) has been created for branch `branch-3.3`
mergify[bot] commented 18 minutes ago

backport branch-3.2

✅ Backports have been created

* [#53144 [Enhancement] Opt memory tracker for FE (backport #53055)](https://github.com/StarRocks/starrocks/pull/53144) has been created for branch `branch-3.2`