This change greatly reduces the likelihood of an error when specifying a large number of property_ids in ga4.combine_property_data().
Fixed indentation
Fixed the following bug
Changed to copy a table for each peoperty_id
Database Error in model base_ga4__events (models/staging/base/base_ga4__events.sql)
The query is too large. The maximum standard SQL query length is 1024.00K characters, including comments and white space characters.
combine_property_data for parallel execution and speedup
My dbt project has more than 40 property_ids.
I want to combine these property_ids in parallel to reduce latency.
Description
I have changed to create a model for each property_id and run a query to execute them in parallel.
This may be a very specific way to implement this, so please let me consult with you to see if there are any problems with this method.
Also, if you have a better way to do this, I would appreciate your advice.
I don't know of any other way to create the model other than creating a large number of int_ga4__combine_property_{{ INDEX }}.
If you have any other good ideas, I would like to improve it.
I've addressed everything except for the parallel execution of combine_property_data in #312.
I'll create a new pull request if I come up with a better approach.
Description & motivation
Motivation
Bugfix
Fix #269.
This change greatly reduces the likelihood of an error when specifying a large number of property_ids in ga4.combine_property_data().
combine_property_data for parallel execution and speedup
My dbt project has more than 40 property_ids. I want to combine these property_ids in parallel to reduce latency.
Description
I have changed to create a model for each property_id and run a query to execute them in parallel. This may be a very specific way to implement this, so please let me consult with you to see if there are any problems with this method. Also, if you have a better way to do this, I would appreciate your advice.
I don't know of any other way to create the model other than creating a large number of
int_ga4__combine_property_{{ INDEX }}
. If you have any other good ideas, I would like to improve it.Execution example
dbt_project.yml
Checklist
dbt test
andpython -m pytest .
to validate existing tests