[HUDI-8556] Add config for max number of colstats columns

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.

https://hudi.apache.org/

Apache License 2.0

5.45k stars 2.42k forks source link

[HUDI-8556] Add config for max number of colstats columns #12310

Open jonvex opened 10 hours ago

jonvex commented 10 hours ago

Change Logs

if user does not specify columns we generate stats for every single column. If they have hundreds or thousands of columns this will have very bad performance.

Impact

config to limit number of colstats generated

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

hudi-bot commented 7 hours ago

CI report:

885b9ff533be8da6c0df730cff351fab332232fa Azure: FAILURE

Bot commands

@hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build