databricks / terraform-provider-databricks

Databricks Terraform Provider
https://registry.terraform.io/providers/databricks/databricks/latest
Other
456 stars 393 forks source link

[Exporter] Use `List` from Go SDK instead of `ListAll` for high volume resources #4119

Closed alexott closed 1 month ago

alexott commented 1 month ago

Some resources, like queries, alerts, dashboards, jobs, DLT pipelines, etc., may have a huge number of resources. Right now, most listing implementations use ListAll from Go SDK, but this leads to a situation when we need to wait until the listing finishes instead of starting processing resources page by page. Switching to the List from Go SDK will allow us to start processing individual resources as soon as we get a page from the backend.