search

hashicorp / terraform-provider-vault

Terraform Vault provider
https://www.terraform.io/docs/providers/vault/
Mozilla Public License 2.0
451 stars 535 forks source link

[Bug]: resource `vault_identity_group_member_group_ids` does not write change? #2195

Open rjhornsby opened 3 months ago

rjhornsby commented 3 months ago

Terraform Core Version

1.7.5

Terraform Vault Provider Version

4.0.0

Vault Server Version

1.15.6

Affected Resource(s)

Expected Behavior

The resource should populate the group membership list, and write this change to vault.

Actual Behavior

The resource appears to execute correctly, but when you plan or apply again TF says that it needs to add the same group_ids to the membership list as it supposedly just added. This happens regardless if you set exclusive to true or false.

If you do your apply, and then change the TF code to intentionally use an empty list, ie member_group_ids = [], TF will say no changes are needed.

This seems to confirm the theory that the list of ids isn't being written/committed to vault properly.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

# this is the target group that will have groups added to it
resource "vault_identity_group" "vault-users" {
  name     = "client-${var.client_abbrev}-vault-users"
  type     = "external"
  policies = concat(var.ex_vault_policies, [vault_policy.vault-users.id])
  lifecycle {
    ignore_changes = [member_entity_ids]
  }
}

# translate names of the additional groups into group_id values
data "vault_identity_group" "extra-groups" {
  for_each   = toset(var.vault_groups)
  group_name = each.value
}

# populate the member group_ids in the target group
resource vault_identity_group_member_group_ids "extra-groups" {
  group_id         = vault_identity_group.vault-users.id
  # edit: jsondecode is not necessary here as of provider 4.x(?)
  member_group_ids = [for group in data.vault_identity_group.extra-groups: group.group_id]
  exclusive        = false
}

I have verified that the group_id that's supposed to be added is a valid, existing vault group. ie it was resolved from the list of group names correctly.

Steps to Reproduce

You can use a simplified version of the above, obtaining valid group_ids of your own and hardcoding it:

resource vault_identity_group_member_group_ids "extra-groups" {
  group_id         = "98765-target-group-id"
  member_group_ids = ["012345-abcd-67890"]
  exclusive        = false
}

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

fairclothjm commented 3 months ago

@rjhornsby Thanks for reporting! Do you only observe this on v4.0.0 or is it also present on v3.X? I don't see any changes to this resource since the major version bump.

rjhornsby commented 3 months ago

@fairclothjm Yep, the issue is present in 3.x and looks like it just carried over into 4.0.0.

rjhornsby commented 3 months ago

I don't know if this is relevant, but trying more to dig into the debug logs I see this:

2024-03-14T13:53:28.965-0500 [WARN]  Provider "registry.terraform.io/hashicorp/vault" produced an unexpected new value for module.client_1234.vault_identity_group_member_group_ids.extra-groups during refresh.
      - .member_group_ids: planned set element cty.StringVal("687aaa25-87ae-55ae-0124-xxxxxxxxx") does not correlate with any element in actual
2024-03-14T13:53:28.968-0500 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"

where 687aaa25-87ae-55ae-0124... is the group_id that's supposed to be getting added to the list. Unfortunately it seems the relevant line stops after the word "actual" so I'm not sure what it's trying to communicate.

It also doesn't seem to stop the provider from deciding there is some kind of change that needs to be applied, but it's hard to tell if the resolved group id value is being thrown out in the above or if I'm not reading the rest of the (massive) debug output properly. Much later in the output, the provider says a change is being made:

2024-03-14T13:53:29.600-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: Updating field "member_group_ids" on Identity Group "be50c371-178b-8f45-70df-xxxxxxxxxxxx": timestamp=2024-03-14T13:53:29.600-0500
2024-03-14T13:53:29.600-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: Reading IdentityGroup be50c371-178b-8f45-70df-xxxxxxxxxxxx from "/identity/group/id/be50c371-178b-8f45-70df-xxxxxxxxxxxx": timestamp=2024-03-14T13:53:29.600-0500
2024-03-14T13:53:29.601-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: Reading Entity from "/identity/group/id/be50c371-178b-8f45-70df-xxxxxxxxxxxx": timestamp=2024-03-14T13:53:29.600-0500
2024-03-14T13:53:29.601-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: [001eb9d9-feda-4aad-a96b-4829fae60430] Vault API Request Details:
---[ REQUEST ]---------------------------------------
GET /v1/identity/group/id/be50c371-178b-8f45-70df-xxxxxxxxxxxx HTTP/1.1
Host: vault.mycorp.com:8200
User-Agent: Go-http-client/1.1
X-Vault-Request: true
X-Vault-Token: hmac-sha256:79d9e453e1a5a4458cd0a590db1fd800572c9d9660eb6e1fadd1c5c0ed6f56b4
Accept-Encoding: gzip

-----------------------------------------------------: timestamp=2024-03-14T13:53:29.601-0500
2024-03-14T13:53:29.686-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: [001eb9d9-feda-4aad-a96b-4829fae60430] Vault API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 200 OK
Content-Length: 970
Cache-Control: no-store
Content-Type: application/json
Date: Thu, 14 Mar 2024 18:53:29 GMT
Strict-Transport-Security: max-age=31536000; includeSubDomains

-----------------------------------------------------: timestamp=2024-03-14T13:53:29.683-0500
2024-03-14T13:53:29.686-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: [e2155fb0-ff26-4de2-b90d-e29e846c000a] Vault API Request Details:
---[ REQUEST ]---------------------------------------
PUT /v1/identity/group/id/be50c371-178b-8f45-70df-xxxxxxxxxxxx HTTP/1.1
Host: vault.mycorp.com:8200
User-Agent: Go-http-client/1.1
Content-Length: 2
X-Vault-Request: true
X-Vault-Token: hmac-sha256:342cb19acfa7929f754a5eb0b1c27d267b7a5549e6bedcb072cd51a638c58b43
Accept-Encoding: gzip

-----------------------------------------------------: timestamp=2024-03-14T13:53:29.686-0500
2024-03-14T13:53:29.779-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: [e2155fb0-ff26-4de2-b90d-e29e846c000a] Vault API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 204 No Content
Content-Length: 0
Cache-Control: no-store
Date: Thu, 14 Mar 2024 18:53:29 GMT
Strict-Transport-Security: max-age=31536000; includeSubDomains

-----------------------------------------------------: timestamp=2024-03-14T13:53:29.777-0500
2024-03-14T13:53:29.779-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: Updated field "member_group_ids" on Identity Group be50c371-178b-8f45-70df-xxxxxxxxxxxx: timestamp=2024-03-14T13:53:29.778-0500
2024-03-14T13:53:29.779-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: Unlocking "/identity/group/id/be50c371-178b-8f45-70df-xxxxxxxxxxxx": timestamp=2024-03-14T13:53:29.778-0500
2024-03-14T13:53:29.779-0500 [DEBUG] provider.terraform-provider-vault_v4.0.0_x5: Unlocked "/identity/group/id/be50c371-178b-8f45-70df-xxxxxxxxxxxx": timestamp=2024-03-14T13:53:29.778-0500
module.client_1234.vault_identity_group_member_group_ids.extra-groups: Modifications complete after 1s [id=be50c371-178b-8f45-70df-xxxxxxxxxxxx]

There's a PUT request in there, but I'm not sure if that output is supposed to contain the resolved group_id or if it's being suppressed as for example POST data would normally(?) not be printed.

fairclothjm commented 3 months ago

@rjhornsby You can set TERRAFORM_VAULT_LOG_BODY to see the request and response body data.

rjhornsby commented 3 months ago

didn't know about those extra env vars. thanks! The group id(s) aren't making it to vault, just an empty set:

---[ REQUEST ]---------------------------------------
PUT /v1/identity/group/id/be50c371-178b-8f45-70df-x HTTP/1.1
Host: vault.mycorp.com:8200
User-Agent: Go-http-client/1.1
Content-Length: 2
X-Vault-Request: true
X-Vault-Token: hmac-sha256:37615f798db4d599a6f806d930f982cf49eadacdaa9c57540b1d0bc47119432e
Accept-Encoding: gzip

{}
rjhornsby commented 3 months ago

I think the error message "does not correlate with any element in actual" is coming from Terraform proper's assertSetValuesCompatible

Forgive me because I'm not a Go programmer, but as best I can tell stumbling and fumbling my way through this the issue is happening because the desired/planned value is a set A (containing the group_id we expect), but the "actual" being compared to here B is nil. B is not an empty set AFAICT, but nil. It makes sense that this might be a problem if you're trying to compare two sets, but one of them is not a set.

Screenshot 2024-03-15 at 13 00 07

Why is B nil? It looks like if the member_group_ids is empty, Vault returns nil instead of an empty list:

╰─○ vault read identity/group/id/be50c371-178b-8f45-70df-xxxxx
...
creation_time        2021-07-12T17:20:37.150135989Z
id                   be50c371-178b-8f45-70df-xxxx
last_update_time     2024-03-15T17:43:36.044116519Z
member_entity_ids    [2c5899c5-a7a4-6ad9-3ad2-xxxxx 36dd8aec-fa70-2642-51d3-xxxxx]
member_group_ids     <nil>

The Vault API responds with a corresponding JSON null to be sure.

I don't know where/how assertSetValuesCompatible is getting called, but IMHO the function is doing its job correctly when it says {"foo"} isn't compatible with nil.

It seems possible solutions are:

fairclothjm commented 3 months ago

@rjhornsby Hello, I think you need to set external_member_group_ids on the vault_identity_group since you are using vault_identity_group_member_group_ids to manage the Group IDs.

I can see the state file has the correct value for member_group_ids under the vault_identity_group_member_group_ids resource when I set external_member_group_ids on vault_identity_group.

rjhornsby commented 3 months ago

thanks @fairclothjm. If I understand what you're saying, the TF should look like:

resource "vault_identity_group" "vault-users" {
  name     = "client-test-vault-users"
  type     = "external"
  lifecycle {
    ignore_changes = [member_entity_ids]
  }
  external_member_group_ids = true  # adds this line
}

resource vault_identity_group_member_group_ids "extra-groups" {
  group_id         = vault_identity_group.vault-users.id
  member_group_ids = ["687aaa25-87ae-55ae-0124-xxxxx"]
  exclusive        = false
}

This doesn't seem to change the behavior. You're right that the member_group_ids is recorded in the state, but TF always wants to apply the change again. Vault doesn't match the TF state:

╰─± vault read -format=json identity/group/id/ff9d2c50-ac30-db3c-5e5e-xxxxx | jq '.member_group_ids'
null

I can still see the same does not correlate with any element in actual in the debug logs. (Which again, to be fair, could be unrelated. I haven't been able to conclusively link this output to the problem because I haven't figured out the code path that gets there.)