Error retrieving IAM policy for pubsub lite topic: Resource not found

baderfahoum17 commented 1 year ago

Hi,

I encountered the issue when I created a pubsub lite topic and a co-responding service account then tried adding it to google_pubsub_topic_iam_member for the topic, terraform throw error saying the topic is not found. and it keeps on failing with subsequent applies

Seem like I might be doing something off that causes the terraform to not find the topic created

I've tried to look for the same issue but I couldn't find anything. If this issue has been reported please direct me to the ticket.

Community Note Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request If you are interested in working on this issue or have submitted a pull request, please leave a comment If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform v0.13.6

Affected Resource(s)

google_pubsub_liteiam*

Terraform Configuration Files

resource "google_pubsub_lite_reservation" "source-reservation" {
  depends_on = [
    google_project_service.enable_api["pubsublite"]
  ]
  project = data.google_project.project.number
  name = "router-source-reservation"
  throughput_capacity = 2
}

resource "google_pubsub_lite_topic" "source-topic" {
  depends_on = [
    google_project_service.enable_api["pubsublite"],
    google_pubsub_lite_reservation.source-reservation
  ]
  project = data.google_project.project.number
  name = "router-source-topic"
  region = var.region

  partition_config {
    count = 1
    capacity {
      publish_mib_per_sec = 4
      subscribe_mib_per_sec = 4
    }
  }

  retention_config {
    per_partition_bytes = 32212254720
  }

  reservation_config {
    throughput_reservation = google_pubsub_lite_reservation.source-reservation.name
  }
}

//# permissions for pubsub publishing for cloudfunctions
resource "google_service_account" "pubsub_publisher" {
  account_id   = "pubsub-publisher"
  display_name = "pubsub-publisher"
}

resource "google_pubsub_topic_iam_member" "publisher" {
  depends_on = [google_pubsub_lite_topic.source-topic]
  topic = google_pubsub_lite_topic.source-topic.name
  role = "roles/pubsublite.publisher"
  member = "serviceAccount:${google_service_account.pubsub_publisher.email}"
}

Debug Output

Error: Error applying plan:

1 error(s) occurred:


Error: Error retrieving IAM policy for pubsub topic "projects/qwiklabs-gcp-02-e8ad4131453d/topics/router-source-topic": googleapi: Error 404: Resource not found (resource=router-source-topic).

  on iam.tf line 22, in resource "google_pubsub_topic_iam_member" "publisher":
  22: resource "google_pubsub_topic_iam_member" "publisher" {

Expected Behavior

terraform should add roles/pubsublite.publisher to the service account 'pubsub-publisher'

steps to reproduce:

terraform apply

b/319231434

baderfahoum17 commented 1 year ago

adding more logs

google_pubsub_lite_reservation.source-reservation: Creating...
google_pubsub_lite_reservation.source-reservation: Creation complete after 0s [id=projects/1020282341613/locations/us-east1/reservations/router-source-reservation]
google_pubsub_lite_topic.source-topic: Creating...
google_pubsub_lite_topic.source-topic: Creation complete after 0s [id=projects/1020282341613/locations/us-east1-c/topics/router-source-topic]
google_pubsub_topic_iam_member.lite_publisher: Creating...
╷
│ Error: Error retrieving IAM policy for pubsub topic "projects/qwiklabs-gcp-00-2e6479aaf8d4/topics/router-source-topic": googleapi: Error 404: Resource not found (resource=router-source-topic).
│

seem like the issue is that the google_pubsub_lite_topic is projecting its name at projects/1020282341613/locations/us-east1-c/topics/router-source-topic which is the topic ID

and the IAM policy is looking for the topic name under: projects/qwiklabs-gcp-00-2e6479aaf8d4/topics/router-source-topic which is only using its Name

edwardmedia commented 1 year ago

@baderfahoum17 you have below line in two resources. Can you try adding it to other two resources to see if the issue is resolved?

 project = data.google_project.project.number

ncoh4gs commented 1 year ago

@edwardmedia I'm a colleague of @baderfahoum17, I've tried to add the line(s) unfortunately with no success

edwardmedia commented 1 year ago

I see where the problem is. The name formats for lite topic and topic are different. The iam resources need to handle the name properly if it is shared for both.

Format is projects/{project}/topics/{topic}.

projects/{project_number}/locations/{location}/topics/{topicId}