chore: recluster disable sort spill

zhyass commented 3 weeks ago

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Briefly describe what this PR aims to solve. Include background context that will help reviewers understand the purpose of the PR.

Fixes #[Link the issue here]

Tests

[ ] Unit Test
[ ] Logic Test
[ ] Benchmark Test
[x] No Test - Explain why

Type of change

[ ] Bug Fix (non-breaking change which fixes an issue)
[ ] New Feature (non-breaking change which adds functionality)
[ ] Breaking Change (fix or feature that could cause existing functionality not to work as expected)
[ ] Documentation Update
[ ] Refactoring
[ ] Performance Improvement
[x] Other (please describe):

This change is

BohuTANG commented 3 weeks ago

I think sort spill not need this setting, but recluster need a setting to disable spill. Because sort spill not only for recluster.

Another issue is if we disable sort spill, the recluster will be OOM? If so the root cause may not to disable spill, but how to avoid spill when recluster?

zhyass commented 2 weeks ago

f we disable sort spill, the recluster will be OOM? If so the root cause may not to disable spill, but how to avoid spill when recluster?

Yes, sort spill will disabled when recluster, others are not affected.

We scaled down the default value of recluster_block_size to minimize the impact of oom.

github-actions[bot] commented 2 weeks ago

Docker Image for PR

tag: pr-15490-116ff0f

note: this image tag is only available for internal use, please check the internal doc for more details.

datafuselabs / databend