tnightengale / dbt-activity-schema

A dbt-Core package for generating models from an activity stream.
GNU General Public License v3.0
38 stars 5 forks source link

Draft: [Patch] Reduce CTE repetition #41

Closed cpita-felix closed 5 months ago

cpita-felix commented 5 months ago

Refactoring macros/dataset.sql to reduce the number of references to the original activity stream CTE/ref passed into it. Should therefore reduce average runtime of the macro as CTE logic/disk reads are performed less frequently.

cpita-felix commented 5 months ago

going to close this PR. My intention was to rewrite dataset.sql as a series of direct left joins between activities, but I realized that that join could explode and generate spurious tuples, which would make it much slower to execute, and would require significant special case handling for the aggregate style relationships.