dbt-labs / terraform-provider-dbtcloud

dbt Cloud Terraform Provider
https://registry.terraform.io/providers/dbt-labs/dbtcloud
MIT License
80 stars 18 forks source link

updating schedule_hours/schedule_interval in a dbtcloud_job does not change in dbt cloud #172

Closed justbldwn closed 11 months ago

justbldwn commented 11 months ago

hi, running into a somewhat strange bug when importing and updating dbtcloud_jobs. we are currently in the process of importing existing dbtcloud_jobs via terraform and looking to standardize a "Generate Docs" job for all of our projects in the process.

when running a plan, we can see a docs job is queued for an updated based on our new module definition:

terraform_docs_update

after running apply, terraform says that the docs job was updated: terraform_docs_updated

if i immediately trigger a new run after the apply finishes, it wants to update the same generate_docs again with the same info as the first screenshot. it wasn't until i manually updated the dbt cloud UI directly with the terraform changes and triggered a new terraform plan for terraform to recognize that the changes were made and stopped trying to update the generate_docs job.

is this something you could look into to see if changing a job schedule_hours/schedule_interval is working properly? i'm not sure if the issue is limited to those fields specifically, or if updating an existing job is just not working properly overall, but wanted to raise this for you to investigate. thanks!

b-per commented 11 months ago

Hi! Thanks for raising an issue here.

Would you be able to share the full content of the job config so I can try to reproduce the issue?

justbldwn commented 11 months ago

sure thing! here is an example we're using to define our job config inside our module:

locals.tf

  project_settings = {
    dev = {
      env_variables = {...}
      environments = [
        {
          name = "Development"
          type = "development"
          env_variables = {...}
        },
        {
          name = "DEV Deployment"
          type = "deployment"
          env_variables = {...}
        },
      ]
    }
    qa = {
      env_variables = {...},
      environments = [
        {
          name = "QA"
          type = "deployment"
          env_variables = {...}
        }
      ]
    }
    prd = {
      env_variables = {...},
      environments = [
        {
          name = "Production"
          type = "deployment"
          env_variables = {...},
        }
      ]
    }
  }

module.tf

resource "dbtcloud_job" "generate_docs" {
  for_each = { for e in local.project_settings[var.env].environments : e.name => e if e.name != "Development" }

  environment_id = dbtcloud_environment.environment[each.value.name].environment_id
  execute_steps = [
    "dbt compile"
  ]
  generate_docs = true
  name          = "Generate Docs"
  project_id    = dbtcloud_project.project.id
  triggers = {
    "custom_branch_only" : true,
    "github_webhook" : false,
    "git_provider_webhook" : false,
    "schedule" : true
  }
  num_threads = 4

  schedule_days     = [0, 1, 2, 3, 4, 5, 6]
  schedule_type     = "days_of_week"
  schedule_interval = 6
}
b-per commented 11 months ago

OK, I am able to reproduce it, which is the first step at getting it fixed! I should have a new version out with a fix either today or tomorrow.