search

facebookincubator / velox

A composable and fully extensible C++ execution engine library for data management systems.
https://velox-lib.io/
Apache License 2.0
3.54k stars 1.17k forks source link

feat(prefix sort): Exclude null byte from sort prefix if column has no nulls #11583

Open zhli1142015 opened 1 week ago

zhli1142015 commented 1 week ago

When inserting data into a RowContainer, we collect information about whether columns contain null values. If a column does not contain nulls, we can omit the null byte in the normalized key. This reduces memory usage and decreases word-by-word comparisons.

For example, with a single sort key of type bigint, without this feature, the normalized key size is 16 bytes (8 + 1 + alignment padding). With this feature, the normalized key size is reduced to 8 bytes. However, this improvement only makes a difference if alignment padding does not negate the optimization. For instance, with a single sort key of type int, the normalized key size remains 8 bytes regardless of whether this feature is enabled.

netlify[bot] commented 1 week ago

Deploy Preview for meta-velox canceled.

Name Link
Latest commit 5fb2b164b5758126c66419c2448765cfbfc2a9f2
Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/6745777732e1c4000813ff0f
zhli1142015 commented 1 week ago

@xiaoxmeng @skadilover @jinchengchenghh could you help to review this PR?