"Provider produced inconsistent final plan" when using `google_cloud_run_v2_job` with a local docker instance

[honungsburk]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to a user, that user is claiming responsibility for the issue.
• Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.9.0 on Macbook Pro M1

• provider registry.terraform.io/hashicorp/google v5.36.0

Affected Resource(s)

• google_cloud_run_v2_job

Terraform Configuration

locals {
  bucket_name = "cross-env-db-backup"
}

terraform {
  required_version = ">=1.6.6"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 5.36.0"
    }
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.2"
    }
  }
  backend "gcs" {}
}
provider "google" {
  project                     = var.project_id
  region                      = var.region
  impersonate_service_account = var.service_account
}

provider "docker" {
  host = "unix:///var/run/docker.sock"
}

resource "google_storage_bucket" "backup" {
  name     = local.bucket_name
  location = var.region
}

resource "google_storage_bucket_iam_member" "bucket_writer" {
  bucket = google_storage_bucket.backup.name
  role   = "roles/storage.objectCreator"
  member = "serviceAccount:${var.service_account}"
}

// Local docker container
resource "docker_container" "copy_service" {
  image = "copy-job"
  name  = "test-copy-job"
}

resource "google_cloud_run_v2_job" "backup_job" {
  provider     = google-beta
  project      = var.project_id
  name         = "cross-env-db-copy"
  location     = var.region
  launch_stage = "BETA"

  template {
    template {
      containers {
        image = docker_container.copy_service.image
        args = [
          "python main.py",
        ]
      }
    }
  }
}

Debug Output

No response

Expected Behavior

I expect it to use the image without issue

Actual Behavior

I get this error:

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for google_cloud_run_v2_job.backup_job to include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/google-beta" produced an invalid new value for .template[0].template[0].containers[0].image: was
│ cty.StringVal("copy-job"), but now cty.StringVal("sha256:6a59e05337dbaf9339d403bafd3f1a82371652e99389d3234096ad92361da264").
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Steps to reproduce

1. terraform apply

Important Factoids

I'm trying to pull a local docker image. Maybe the docker provider finds it but the google provider try to pull it from docker hub?

References

No response

[ggtisc]

Hi @honungsburk!

Checking the google_cloud_run_v2_job Google cloud service we test the functionality with this terraform registry template and the result was successful without errors.

I suggest you to check your project configuration, environment variables and your docker_container image as it is a provider we have no control over.

You can guide yourself with this other example, wich was used in this ticket:

provider "google" {
  user_project_override = true
  billing_project = "my-project"
  project = "my-project"
}

terraform {
  required_providers {
    google = {
        source = "hashicorp/google-beta"
        version = "5.36.0"
    }
  }
}

resource "google_cloud_run_v2_job" "cr_v2_job_18625" {
  provider = google-beta
  name         = "cr-v2-job-18625"
  location     = "us-central1"
  launch_stage = "BETA"

  template {
    template {
      containers {
        image = "us-docker.pkg.dev/cloudrun/container/job"
      }
    }
  }
}

[dibunker]

@honungsburk I think your expected behavior is incorrect and this isn't a bug, but a misconfigured terraform and thus you should expect a failure.

As the Cloud Run Job documentation mentions, a Cloud Run Job must pull images from either an Artifact Registry or Docker Hub. The image argument reference documentation also reiterates this requirement.

image - (Required) URL of the Container image in Google Container Registry or Google Artifact Registry. More info: https://kubernetes.io/docs/concepts/containers/images

To correct your configuration, you should follow the steps for deploying local sources to cloud run in your terraform:

1. Create an artifact registry
2. Push your local image to the new registry
3. In your cloud run job, use the URI that points to the image in the new registry.

You will most likely have to use the depends_on terraform meta argument to ensure those resources are created in the proper order.

[honungsburk]

Yes, eventually I was able to figure out that you have to upload the images to some repository available through the internet. Would it be possible to guide the user by returning a more informative error message?

[zli82016]

@honungsburk , I don't think it is possible to guide the user by returning a more informative error message.

[ggtisc]

@honungsburk you just need to follow the provided guides and read the documentation. Doing that is a good practice before starting to use any software.