search

dbt-labs / terraform-provider-dbtcloud

dbt Cloud Terraform Provider
https://registry.terraform.io/providers/dbt-labs/dbtcloud
MIT License
80 stars 18 forks source link

fix: Update GetGroup function to get by id rather than list and find #197

Closed SudoSpartanDan closed 9 months ago

SudoSpartanDan commented 9 months ago

We found that once you get over 100 groups in DBT, the GetGroup function was failing to find some group. This is because the function uses the list endpoint, which is paged and defaults to 100 groups as the limit to return.

This PR adjusts the function to use the Retrieve Group endpoint: https://docs.getdbt.com/dbt-cloud/api-v3#/operations/Retrieve%20Group

For some reason, the API docs say data returns as a list but actually testing it, it came back as a JSON. #shrug, I think it's probably a doc that just needs updating.

b-per commented 9 months ago

Thanks for this!

There are actually different parts in the provider where things might break when we reach more than 100 items. I will create a new issue to track those and either try to switch to an endpoint that queries one specific ID or loop through the results pages.

I will release a new version tomorrow I think with this and maybe a couple of extra changes.