Changing job's `project_id` attribute does not force replacement

[anaghshineh]

Describe the bug

Hello! We ran into a weird issue recently where we updated the project_id and environment_id attributes associated with a pre-existing dbt_cloud_job resource. Instead of doing a replacement (destroy + create), Terraform tried to do an update-in-place operation, which seemed odd.

Error message

When attempting an apply, we got the following 404 error:

│ Error: POST url: https://cloud.getdbt.com/api/v2/accounts/<account-id-redacted>/jobs/<job-id-redacted>/, status: 404, body: {"status":{"code":404,"is_success":false,"user_message":"The requested resource was not found. Please check that you have the proper permissions. If you are seeing this in error, please contact support!","developer_message":""},"data":null}

Resource configuration

resource "dbt_cloud_job" "transform_job" {

  environment_id       = var.environment_id
  execute_steps        = ["dbt build --select ${join(" ", [for table in var.tables : "+${table}"])}"]
  generate_docs        = false
  is_active            = true
  name                 = var.name
  num_threads          = var.threads
  project_id           = var.project_id
  run_generate_sources = false

  target_name = var.env
  triggers = {
    "git_provider_webhook" : false,
    "custom_branch_only" : false,
    "github_webhook" : false,
    "schedule" : false
  }

  # These parameters are required. The current dbt provider and API docs
  # indicate they are optional, but terraform apply will fail without them. 
  schedule_days = [0, 1, 2, 3, 4, 5, 6]
  schedule_type = "days_of_week"
}

Expected behavior

I would expect Terraform to do a replacement of the job in this instance. Delete it from the project in which it was originally created & then create it in the new project associated with the new project_id value.

I think apply failed because it was trying to locate the job in the new project and couldn't find it, since it still existed in the old project??

Config (please complete the following information):

( the version can be retrieved running the command terraform providers )

• dbt Cloud provider version: "0.2.6"

Additional context

Add any other context about the problem here.

[b-per]

Thanks. I will update the provider to force a new job when the project changes. I tried changing the environment for a given project, and this works.

[b-per]

As a side note, are you still using the dbt_cloud_xxx resources? Since 0.3 I removed support for those and only allow dbtcloud_xxx.

I was thinking of doing the change for this bug only be released for 0.3.x. I could also do a new 0.2.x release as well for this one (if you need it), but this is not something I want to support on the long run.

[anaghshineh]

Hey @b-per ! Thanks, as always, for such speedy & thorough responses! Appreciate you.

Yea, we're still using an instance of the dbt_cloud_xxx resource. I'd like to upgrade our provider version & move to dbtcloud_xxx. With that in mind – I feel good about your making this change for 0.3.x without touching prior versions 😄.

THANK YOU!

[anaghshineh]

Hey, @b-per – thought about this a little more. I actually think it'd be great to have the change apply to 0.2.x as well. If I try to do the upgrade to 0.3.x and dbt_cloud_xxx => dbtcloud_xxx, it'll fail for my platform's various tenants, since the newer version got rid of dbt_cloud_xxx.

That way, I can upgrade to the latest version for 0.2.x & change to dbtcloud_xxx and then in a later phase upgrade to 0.3.x.

[b-per]

Understood.

I released 0.2.26 with this fix and will add it to the next 0.3.x as well. 0.2.26 should show up on the registry in a few minutes.