cube-js / cube

📊 Cube — Universal semantic layer platform for AI, BI, spreadsheets, and embedded analytics
https://cube.dev
Other
17.96k stars 1.78k forks source link

fix(cubestore): don't add all measures to default preagg index #8789

Closed KSDaemon closed 1 month ago

KSDaemon commented 1 month ago

This change improves the cube store preaggregation creation: the default index for a preaggregation now excludes all measures from the sorting keys

Check List

vercel[bot] commented 1 month ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

8 Skipped Deployments | Name | Status | Preview | Comments | Updated (UTC) | | :--- | :----- | :------ | :------- | :------ | | **examples-angular-dashboard** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-angular-dashboard/5f5xD8Hbx6T6BtxGAURx6R8Y6mfn)) | [Visit Preview](https://examples-angular-dashboard-git-cube-store-no-de-b0b0b6-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-react-d3** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-react-d3/5KtaJPutb843Jq612h7bdyc1e6VB)) | [Visit Preview](https://examples-react-d3-git-cube-store-no-deflt-idx-measures-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-react-dashboard** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-react-dashboard/AjejCVHCBLoPLEu41rCGKaZbA8J9)) | [Visit Preview](https://examples-react-dashboard-git-cube-store-no-defl-d66d5d-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-react-data-table** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-react-data-table/DXB1mpaxkZjwt77yuSdJvuopww7x)) | [Visit Preview](https://examples-react-data-table-git-cube-store-no-def-303780-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-react-highcharts** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-react-highcharts/GmHMWeDfs5HicHZWt2k4A8cpgMbp)) | [Visit Preview](https://examples-react-highcharts-git-cube-store-no-def-b878cc-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-react-material-ui** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-react-material-ui/BU3XfGfuauCszoF94qZgLHQNXyMB)) | [Visit Preview](https://examples-react-material-ui-git-cube-store-no-de-066b5e-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-react-pivot-table** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-react-pivot-table/5Pm5xP2JZU8fCJUUFgLDNKXmCBGF)) | [Visit Preview](https://examples-react-pivot-table-git-cube-store-no-de-b22b51-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm | | **examples-vue-query-builder** | ⬜️ Ignored ([Inspect](https://vercel.com/cube-dev/examples-vue-query-builder/4dzpJZEGmW3JHc8QMnbyTcNkv8C4)) | [Visit Preview](https://examples-vue-query-builder-git-cube-store-no-de-75cd66-cube-dev.vercel.app) | | Oct 9, 2024 2:45pm |
igorlukanin commented 1 month ago

@KSDaemon I believe this has to be changed: https://cube.dev/docs/product/caching/using-pre-aggregations#using-indexes

Should this be considered a breaking change? Also, will upgrading to this version result in pre-aggregations being rebuilt, entirely or partially?

KSDaemon commented 1 month ago

@igorlukanin Actually No :) It's internal optimization, so indexes are still copies of all preagg data. The difference is that now, sorting keys for the index do not include measures (as it's mostly useless, no filtering is done by measures, potentially they may be useful for postagregation filtering (aka HAVING xxx). But that is another story.

Rgd existing indexes — there should be no changes. This technique will be applied only for new index creation.

igorlukanin commented 1 month ago

Docs still need to be updated wrt what is included in the default index, don't they?

KSDaemon commented 1 month ago

When you define a pre-aggregation without any explicit indexes, the default index is created. In this index, dimensions come first, time dimensions come second, and measures come last.

This is still true :)

igorlukanin commented 1 month ago

@KSDaemon Hopefully I did it correctly.