[BugFix]remove check on num_rows and avail levels, batch skip to avoid big memory alloc (backport #47544)

[mergify[bot]]

Why I'm doing:

to avoid big memory alloc, level_decoder return 1024*1024 level at most, so row_nums > avail_level is not suitable.

And to avoid big memory alloc, we'd better skip batch by batch(8096 in code).

What I'm doing:

Fixes #issue

What type of PR is this:

• [x] BugFix
• [ ] Feature
• [ ] Enhancement
• [ ] Refactor
• [ ] UT
• [ ] Doc
• [ ] Tool

Does this PR entail a change in behavior?

• [ ] Yes, this PR will result in a change in behavior.
• [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
• [ ] Parameter changes: default values, similar parameters but with different default values
• [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
• [ ] Feature removed
• [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• [ ] I have added test cases for my bug fix or my new feature
• [ ] This pr needs user documentation (for new or modified features or behaviors)
  • [ ] I have added documentation for my new feature or new function
• [x] This is a backport pr

Bugfix cherry-pick branch check:

• [x] I have checked the version labels which the pr will be auto-backported to the target branch
  • [x] 3.3
  • [ ] 3.2
  • [ ] 3.1
  • [ ] 3.0
  • [ ] 2.5

    ---

    This is an automatic backport of pull request #47544 done by Mergify.

    Why I'm doing:

    to avoid big memory alloc, level_decoder return 1024*1024 level at most, so row_nums > avail_level is not suitable.

And to avoid big memory alloc, we'd better skip batch by batch(8096 in code).

What I'm doing:

Fixes #issue

What type of PR is this:

• [x] BugFix
• [ ] Feature
• [ ] Enhancement
• [ ] Refactor
• [ ] UT
• [ ] Doc
• [ ] Tool

Does this PR entail a change in behavior?

• [ ] Yes, this PR will result in a change in behavior.
• [x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
• [ ] Parameter changes: default values, similar parameters but with different default values
• [ ] Policy changes: use new policy to replace old one, functionality automatically enabled
• [ ] Feature removed
• [ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• [ ] I have added test cases for my bug fix or my new feature
• [ ] This pr needs user documentation (for new or modified features or behaviors)
  • [ ] I have added documentation for my new feature or new function
• [x] This is a backport pr

[zombee0]

https://github.com/Mergifyio close

[mergify[bot]]

close

❌ Sorry but I didn't understand the command. Please consult the commands documentation 📚.