ccao-data / data-architecture

Codebase for CCAO data infrastructure construction and management
https://ccao-data.github.io/data-architecture/
5 stars 3 forks source link

Bump number of threads used by all dbt targets #498

Closed dfsnow closed 3 weeks ago

dfsnow commented 3 weeks ago

The dbt threads argument specifies the maximum number of models that can be simultaneously built without breaking the dependency graph. We had it set to 5 for reasons I can't quite remember, but Athena can handle many more simultaneous queries (DDL statements, in this case). This PR bumps the maximum number of threads to 16. This should result in slightly faster full DAG builds.