Folder level audit configs don't work.

mikhail-khodorovskiy commented 5 years ago

resource "google_folder_iam_policy" "audit_logs" {
  folder = "${google_folder.enironment.id}"
  policy_data = "${data.google_iam_policy.audit_log_configs.policy_data}"
}

data "google_iam_policy" "audit_log_configs" {

  binding {
    role = "roles/owner"

    members = [
      "${var.service_account_role}"
    ]
  }

  audit_config {
    service = "cloudkms.googleapis.com"

    audit_log_configs {
      log_type = "DATA_READ"
    }

    audit_log_configs {
      log_type = "DATA_WRITE"
    }

    audit_log_configs {
      log_type = "ADMIN_READ"
    }
  }
}

After the template is run the cloudkms service does not have audit configs turn on in the GCP Web console. On subsequent runs, the change is detected and tried to be applied but it never succeeds.

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment
If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Affected Resource(s)

google_XXXXX

Terraform Configuration Files

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key: https://www.hashicorp.com/security
# If reproducing the bug involves modifying the config file (e.g., apply a config,
# change a value, apply the config again, see the bug) then please include both the
# version of the config before the change, and the version of the config after the change.

Debug Output

Panic Output

Expected Behavior

Actual Behavior

Steps to Reproduce

terraform apply

Important Factoids

References

0000

rileykarson commented 5 years ago

Are you able to share debug logs? Feel free to strip specific folder IDs, but knowing the specific HTTP requests made can be a big help in cases like this.

mikhail-khodorovskiy commented 5 years ago


fTransformer: Module: UPDATE: google_folder_iam_policy.audit_logs
  policy_data: "{\"bindings\":[{\"members\":[\"group:GCPAdmins@cambiahealthplan.com\"],\"role\":\"roles/editor\"},{\"members\":[\"group:Tilikum@cambiahealthplan.com\"],\"role\":\"roles/logging.viewer\"},{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"},{\"members\":[\"group:Tilikum@cambiahealthplan.com\"],\"role\":\"roles/storage.objectViewer\"},{\"members\":[\"group:Tilikum@cambiahealthplan.com\"],\"role\":\"roles/viewer\"}]}" => "{\"auditConfigs\":[{\"auditLogConfigs\":[{\"logType\":\"DATA_WRITE\"},{\"logType\":\"DATA_READ\"},{\"logType\":\"ADMIN_READ\"}],\"service\":\"cloudkms.googleapis.com\"}],\"bindings\":[{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"}]}"
2019/05/31 14:11:22 [TRACE] DiffTransformer: Resource "google_folder_iam_policy.audit_logs": *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"policy_data":*terraform.ResourceAttrDiff{Old:"{\"bindings\":[{\"members\":[\"group:GCPAdmins@cambiahealthplan.com\"],\"role\":\"roles/editor\"},{\"members\":[\"group:Tilikum@cambiahealthplan.com\"],\"role\":\"roles/logging.viewer\"},{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"},{\"members\":[\"group:Tilikum@cambiahealthplan.com\"],\"role\":\"roles/storage.objectViewer\"},{\"members\":[\"group:Tilikum@cambiahealthplan.com\"],\"role\":\"roles/viewer\"}]}", New:"{\"auditConfigs\":[{\"auditLogConfigs\":[{\"logType\":\"DATA_WRITE\"},{\"logType\":\"DATA_READ\"},{\"logType\":\"ADMIN_READ\"}],\"service\":\"cloudkms.googleapis.com\"}],\"bindings\":[{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"}]}", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
2019/05/31 14:11:22 [TRACE] Dif```

Is this what you are looking for?

rileykarson commented 5 years ago

Yep! If you're able to share more of the surrounding lines, or the entire log, that would be a big help.

mikhail-khodorovskiy commented 5 years ago

2019/05/31 17:57:53 [TRACE] dag/walk: walking "root"
2019/05/31 17:57:53 [TRACE] vertex 'root.root': walking
2019/05/31 17:57:53 [INFO] terraform: building graph: GraphTypeApply
2019/05/31 17:57:53 [TRACE] DiffTransformer: starting
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: UPDATE: google_folder_iam_policy.audit_logs
  policy_data: "{\"bindings\":[{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"}]}" => "{\"auditConfigs\":[{\"auditLogConfigs\":[{\"logType\":\"DATA_WRITE\"},{\"logType\":\"DATA_READ\"},{\"logType\":\"ADMIN_READ\"}],\"service\":\"cloudkms.googleapis.com\"}],\"bindings\":[{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"}]}"
2019/05/31 17:57:53 [TRACE] DiffTransformer: Resource "google_folder_iam_policy.audit_logs": *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"policy_data":*terraform.ResourceAttrDiff{Old:"{\"bindings\":[{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"}]}", New:"{\"auditConfigs\":[{\"auditLogConfigs\":[{\"logType\":\"DATA_WRITE\"},{\"logType\":\"DATA_READ\"},{\"logType\":\"ADMIN_READ\"}],\"service\":\"cloudkms.googleapis.com\"}],\"bindings\":[{\"members\":[\"serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com\"],\"role\":\"roles/owner\"}]}", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: CREATE: google_folder_iam_binding.folder_iam_authoritative.0
  etag:               "" => "<computed>"
  folder:             "" => "folders/521340977610" (forces new resource)
  members.#:          "" => "1"
  members.1284695358: "" => "group:Tilikum@cambiahealthplan.com"
  role:               "" => "roles/viewer" (forces new resource)
CREATE: google_folder_iam_binding.folder_iam_authoritative.1
  etag:               "" => "<computed>"
  folder:             "" => "folders/521340977610" (forces new resource)
  members.#:          "" => "1"
  members.1284695358: "" => "group:Tilikum@cambiahealthplan.com"
  role:               "" => "roles/storage.objectViewer" (forces new resource)
CREATE: google_folder_iam_binding.folder_iam_authoritative.2
  etag:               "" => "<computed>"
  folder:             "" => "folders/521340977610" (forces new resource)
  members.#:          "" => "1"
  members.1284695358: "" => "group:Tilikum@cambiahealthplan.com"
  role:               "" => "roles/logging.viewer" (forces new resource)
CREATE: google_folder_iam_binding.folder_iam_authoritative.3
  etag:               "" => "<computed>"
  folder:             "" => "folders/521340977610" (forces new resource)
  members.#:          "" => "2"
  members.161973065:  "" => "group:SecurityAdmins@cambiahealthplan.com"
  members.2545438266: "" => "group:GCPAdmins@cambiahealthplan.com"
  role:               "" => "roles/editor" (forces new resource)
2019/05/31 17:57:53 [TRACE] DiffTransformer: Resource "google_folder_iam_binding.folder_iam_authoritative.2": *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"etag":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "folder":*terraform.ResourceAttrDiff{Old:"", New:"folders/521340977610", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, "id":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x2}, "members.#":*terraform.ResourceAttrDiff{Old:"", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "members.1284695358":*terraform.ResourceAttrDiff{Old:"", New:"group:Tilikum@cambiahealthplan.com", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "role":*terraform.ResourceAttrDiff{Old:"", New:"roles/logging.viewer", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
2019/05/31 17:57:53 [TRACE] DiffTransformer: Resource "google_folder_iam_binding.folder_iam_authoritative.3": *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"etag":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "folder":*terraform.ResourceAttrDiff{Old:"", New:"folders/521340977610", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, "id":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x2}, "members.#":*terraform.ResourceAttrDiff{Old:"", New:"2", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "members.161973065":*terraform.ResourceAttrDiff{Old:"", New:"group:SecurityAdmins@cambiahealthplan.com", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "members.2545438266":*terraform.ResourceAttrDiff{Old:"", New:"group:GCPAdmins@cambiahealthplan.com", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "role":*terraform.ResourceAttrDiff{Old:"", New:"roles/editor", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
2019/05/31 17:57:53 [TRACE] DiffTransformer: Resource "google_folder_iam_binding.folder_iam_authoritative.1": *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"etag":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "folder":*terraform.ResourceAttrDiff{Old:"", New:"folders/521340977610", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, "id":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x2}, "members.#":*terraform.ResourceAttrDiff{Old:"", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "members.1284695358":*terraform.ResourceAttrDiff{Old:"", New:"group:Tilikum@cambiahealthplan.com", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "role":*terraform.ResourceAttrDiff{Old:"", New:"roles/storage.objectViewer", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
2019/05/31 17:57:53 [TRACE] DiffTransformer: Resource "google_folder_iam_binding.folder_iam_authoritative.0": *terraform.InstanceDiff{mu:sync.Mutex{state:0, sema:0x0}, Attributes:map[string]*terraform.ResourceAttrDiff{"etag":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "folder":*terraform.ResourceAttrDiff{Old:"", New:"folders/521340977610", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}, "id":*terraform.ResourceAttrDiff{Old:"", New:"", NewComputed:true, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x2}, "members.#":*terraform.ResourceAttrDiff{Old:"", New:"1", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "members.1284695358":*terraform.ResourceAttrDiff{Old:"", New:"group:Tilikum@cambiahealthplan.com", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:false, Sensitive:false, Type:0x0}, "role":*terraform.ResourceAttrDiff{Old:"", New:"roles/viewer", NewComputed:false, NewRemoved:false, NewExtra:interface {}(nil), RequiresNew:true, Sensitive:false, Type:0x0}}, Destroy:false, DestroyDeposed:false, DestroyTainted:false, Meta:map[string]interface {}(nil)}
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] DiffTransformer: Module: 
2019/05/31 17:57:53 [TRACE] Graph after step *terraform.DiffTransformer:

module.janus_environment_folder.google_folder_iam_policy.audit_logs - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3] - *terraform.NodeApplyableResource
2019/05/31 17:57:53 [TRACE] Graph after step *terraform.OrphanOutputTransformer:

module.janus_environment_folder.google_folder_iam_policy.audit_logs - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3] - *terraform.NodeApplyableResource
2019/05/31 17:57:53 [TRACE] AttachResourceConfigTransformer: Beginning...
2019/05/31 17:57:53 [TRACE] AttachResourceConfigTransformer: Attach resource config request: module.janus_environment_folder.google_folder_iam_policy.audit_logs
2019/05/31 17:57:53 [TRACE] Attaching resource config: &config.Resource{Mode:0, Name:"audit_logs", Type:"google_folder_iam_policy", RawCount:(*config.RawConfig)(0xc00015ad90), RawConfig:(*config.RawConfig)(0xc00015ac40), Provisioners:[]*config.Provisioner(nil), Provider:"", DependsOn:[]string(nil), Lifecycle:config.ResourceLifecycle{CreateBeforeDestroy:false, PreventDestroy:false, IgnoreChanges:[]string(nil)}}
2019/05/31 17:57:53 [TRACE] AttachResourceConfigTransformer: Attach resource config request: module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2]
2019/05/31 17:57:53 [TRACE] Attaching resource config: &config.Resource{Mode:0, Name:"folder_iam_authoritative", Type:"google_folder_iam_binding", RawCount:(*config.RawConfig)(0xc0001df500), RawConfig:(*config.RawConfig)(0xc0001df490), Provisioners:[]*config.Provisioner(nil), Provider:"", DependsOn:[]string(nil), Lifecycle:config.ResourceLifecycle{CreateBeforeDestroy:false, PreventDestroy:false, IgnoreChanges:[]string(nil)}}
2019/05/31 17:57:53 [TRACE] AttachResourceConfigTransformer: Attach resource config request: module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3]
2019/05/31 17:57:53 [TRACE] Attaching resource config: &config.Resource{Mode:0, Name:"folder_iam_authoritative", Type:"google_folder_iam_binding", RawCount:(*config.RawConfig)(0xc0001df500), RawConfig:(*config.RawConfig)(0xc0001df490), Provisioners:[]*config.Provisioner(nil), Provider:"", DependsOn:[]string(nil), Lifecycle:config.ResourceLifecycle{CreateBeforeDestroy:false, PreventDestroy:false, IgnoreChanges:[]string(nil)}}
2019/05/31 17:57:53 [TRACE] AttachResourceConfigTransformer: Attach resource config request: module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1]
2019/05/31 17:57:53 [TRACE] Attaching resource config: &config.Resource{Mode:0, Name:"folder_iam_authoritative", Type:"google_folder_iam_binding", RawCount:(*config.RawConfig)(0xc0001df500), RawConfig:(*config.RawConfig)(0xc0001df490), Provisioners:[]*config.Provisioner(nil), Provider:"", DependsOn:[]string(nil), Lifecycle:config.ResourceLifecycle{CreateBeforeDestroy:false, PreventDestroy:false, IgnoreChanges:[]string(nil)}}
2019/05/31 17:57:53 [TRACE] AttachResourceConfigTransformer: Attach resource config request: module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0]
2019/05/31 17:57:53 [TRACE] Attaching resource config: &config.Resource{Mode:0, Name:"folder_iam_authoritative", Type:"google_folder_iam_binding", RawCount:(*config.RawConfig)(0xc0001df500), RawConfig:(*config.RawConfig)(0xc0001df490), Provisioners:[]*config.Provisioner(nil), Provider:"", DependsOn:[]string(nil), Lifecycle:config.ResourceLifecycle{CreateBeforeDestroy:false, PreventDestroy:false, IgnoreChanges:[]string(nil)}}
2019/05/31 17:57:53 [TRACE] Graph after step *terraform.AttachResourceConfigTransformer:

module.janus_environment_folder.google_folder_iam_policy.audit_logs - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3] - *terraform.NodeApplyableResource
2019/05/31 17:57:53 [DEBUG] Attaching resource state to "module.janus_environment_folder.google_folder_iam_policy.audit_logs": &terraform.ResourceState{Type:"google_folder_iam_policy", Dependencies:[]string{"data.google_iam_policy.audit_log_configs", "google_folder.enironment"}, Primary:(*terraform.InstanceState)(0xc0006f0410), Deposed:[]*terraform.InstanceState{}, Provider:"provider.google", mu:sync.Mutex{state:0, sema:0x0}}
2019/05/31 17:57:53 [DEBUG] Resource state not found for "module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2]": module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2]
2019/05/31 17:57:53 [DEBUG] Resource state not found for "module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3]": module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3]
2019/05/31 17:57:53 [DEBUG] Resource state not found for "module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1]": module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1]
2019/05/31 17:57:53 [DEBUG] Resource state not found for "module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0]": module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0]
2019/05/31 17:57:53 [TRACE] Graph after step *terraform.AttachStateTransformer:

module.janus_environment_folder.google_folder_iam_policy.audit_logs - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[0] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[1] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[2] - *terraform.NodeApplyableResource
module.janus_environment_folder.module.folder_iam_binding.google_folder_iam_binding.folder_iam_authoritative[3] - *terraform.NodeApplyableResource

This is from another run so the dates won't match.

rileykarson commented 5 years ago

Specifically, the HTTP request + response pairs would be helpful. That partial log doesn't include them.

mikhail-khodorovskiy commented 5 years ago

I am using TRACE debug level and they are no requests/responses logged. Do you want me to get the HTTP logs using GCP APIs? I can't get any Logs from APIs on the folder level - any idea where I can find these?

mikhail-khodorovskiy commented 5 years ago

the same exact setup works for the project iam policy just fine:

resource "google_project_iam_policy" "audit_logs" {
  project = "${module.host-project.project_id}"
  policy_data = "${data.google_iam_policy.audit_log_configs.policy_data}"
}

data "google_iam_policy" "audit_log_configs" {

  binding {
    role = "roles/owner"

    members = [
      "serviceAccount:service-terraform-admin@x25-terraform-admin.iam.gserviceaccount.com"
    ]
  }

  audit_config {
    service = "cloudkms.googleapis.com"

    audit_log_configs {
      log_type = "DATA_READ"
    }

    audit_log_configs {
      log_type = "DATA_WRITE"
    }

    audit_log_configs {
      log_type = "ADMIN_READ"
    }
  }
}

rileykarson commented 5 years ago

Can you try DEBUG level? My apologies, I thought based on this:

2019/05/31 17:57:53 [DEBUG] Resource state not found

you were @ debug, but I think that's from Terraform core which handles this stuff a little differently.

mikhail-khodorovskiy commented 5 years ago

2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: ---[ REQUEST ]---------------------------------------
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: POST /v1/projects/hsq-dev-pipeline-a24c:getIamPolicy?alt=json&prettyPrint=false HTTP/1.1
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Host: cloudresourcemanager.googleapis.com
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: User-Agent: google-api-go-client/0.5 Terraform/0.12.0 (+https://www.terraform.io) terraform-provider-google/2.7.0
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Content-Length: 3
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Content-Type: application/json
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Accept-Encoding: gzip
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: 
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: 
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: -----------------------------------------------------
2019-06-03T12:47:58.950-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: 2019/06/03 12:47:58 [DEBUG] Locking "iam-project-hsq-dev-pipeline-a24c"
2019-06-03T12:47:59.138-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: 2019/06/03 12:47:59 [DEBUG] Google API Response Details:
2019-06-03T12:47:59.138-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: ---[ RESPONSE ]--------------------------------------
2019-06-03T12:47:59.138-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: HTTP/2.0 200 OK
2019-06-03T12:47:59.138-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
2019-06-03T12:47:59.138-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Cache-Control: private
2019-06-03T12:47:59.138-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Content-Type: application/json; charset=UTF-8
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Date: Mon, 03 Jun 2019 19:47:59 GMT
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Server: ESF
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Server-Timing: gfet4t7; dur=168
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Vary: Origin
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Vary: X-Origin
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: Vary: Referer
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: X-Content-Type-Options: nosniff
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: X-Frame-Options: SAMEORIGIN
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: X-Xss-Protection: 0
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: 
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: {
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:  "version": 1,
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:  "etag": "BwWKcJK4lfQ=",
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:  "bindings": [
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "organizations/696386872137/roles/RedlockViewerSA",
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:hsq-dev-redlock@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "projects/hsq-dev-pipeline-a24c/roles/composer.WorkerCustom",
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:composer-worker-sa@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/bigquery.dataEditor",
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:composer-worker-sa@hsq-dev-pipeline-a24c.iam.gserviceaccount.com",
12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/cloudkms.admin",
2019-06-03T12:47:59.139-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/cloudkms.cryptoKeyEncrypter",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/composer.environmentAndStorageObjectAdmin",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:composer-worker-sa@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/composer.user",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/composer.worker",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:composer-worker-sa@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/compute.securityAdmin",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:hsq-dev-redlock@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/compute.viewer",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:hsq-dev-datadog@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/dataflow.admin",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:composer-worker-sa@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/dataflow.worker",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:dataflow-worker@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/datastore.importExportAdmin",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:composer-worker-sa@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/deploymentmanager.editor",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/editor",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:1002949831369@cloudservices.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/iam.serviceAccountAdmin",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:1002949831369@cloudservices.gserviceaccount.com",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/monitoring.viewer",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:hsq-dev-datadog@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/resourcemanager.projectIamAdmin",
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.140-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/storage.admin",
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:janus-google@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/storage.objectAdmin",
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:dataflow-worker@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/storage.objectCreator",
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:dataflow-worker@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/storage.objectViewer",
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:dataflow-worker@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "role": "roles/viewer",
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "members": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     "serviceAccount:hsq-dev-redlock@hsq-dev-pipeline-a24c.iam.gserviceaccount.com"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   }
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:  ],
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:  "auditConfigs": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "service": "cloudkms.googleapis.com",
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    "auditLogConfigs": [
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:      "logType": "DATA_WRITE"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:      "logType": "DATA_READ"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     },
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     {
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:      "logType": "ADMIN_READ"
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:     }
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:    ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:   }
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4:  ]
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: }
2019-06-03T12:47:59.141-0700 [DEBUG] plugin.terraform-provider-google_v2.7.0_x4: -----------------------------------------------------

And BTW, once I removed the the project level audit logs config, it still persists which is a different but related issue.

mikhail-khodorovskiy commented 5 years ago

Just to make sure - the status IS not waiting for response and I just verified that the issue is still valid.

rileykarson commented 5 years ago

Yep, sorry! I haven't had time to dig in yet.

emailbob commented 5 years ago

I think the google_folder_iam_policy resource was missed when google_project_iam_policy was updated to support audit config. See this PR https://github.com/terraform-providers/terraform-provider-google/pull/2731

rileykarson commented 5 years ago

Yup, those were my findings as well! After looking into it it seems that we never added general support for audit configs, only adding them to google_project_iam_policy. Right now, reading audit configs works in no but project APIs and adding them works in some older ones.

There's no documentation to suggest that they're not supported in cases other than project, so this leaves us in a bit of a weird place as to whether this is a bug with a feature we should have or a feature we don't support. Sorry about that!

It may count as a breaking change to properly support them, which would delay the feature until our next major release. Since users could have them defined out of band, they'll see a diff when performing a minor version update. Normally Terraform gives us some ability to handle this case, but because the field on the policy resource is a JSON string, Terraform's built in schema tooling is limited.

We're thinking about how we can add support for the feature to the _policy resource while minimising the disruption for our users.

leighmhart commented 5 years ago

@rileykarson et al, this is also an issue for folks using google_project_iam_member to assign IAM policy to their projects - in their case, the only way they could implement audit_policy would be to ensure that the google_project_iam_binding does not use any roles that are in use with google_project_iam_member per the documentation https://www.terraform.io/docs/providers/google/r/google_project_iam.html

huron25 commented 5 years ago

@allandrick - that's where I am stuck. I'm trying to implement some compliance controls at the folder level since I don't want to implement the audit policy at the project and blow away the IAM permissions already present (mostly worried about the service accounts for Google APIs and compute engine).

Is there an ETA on whichever fix terraform is choosing (or code I can use locally to fix the folder IAM policy?

cschroer commented 5 years ago

I would consider this as a bug, as by GCPs docs auditConfig is supported on org, folder and project level. If those ressources (org and folder mainly) do not yet support them yet, this should be added.

I also tried to enforce some "global" audit logging today and ran into this. I just want to set it on a folder level, to avoid project owners to disable it again ;)

chrisst commented 4 years ago

Assigning to Hashibot as @ondrejklucka has generously already started on the solution for this.

chrisst commented 4 years ago

After digging a bit more it looks like the folder_iam_policy resource was fixed to correctly read/set audit_config as part of https://github.com/GoogleCloudPlatform/magic-modules/pull/2313

Closing this bug as resolved. I'll work to get the audit_config resource PR merged in separately.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

hashicorp / terraform-provider-google