search

hashicorp / terraform-provider-vault

Terraform Vault provider
https://www.terraform.io/docs/providers/vault/
Mozilla Public License 2.0
465 stars 542 forks source link

Segmentation fault in auth role/team when the auth method is deleted #1720

Open kingledion opened 1 year ago

kingledion commented 1 year ago

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

1.3.3 Vault Provider 3.12.0

Affected Resource(s)

Please list the resources as a list, for example:

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

resource "vault_kubernetes_auth_backend_role" "tools_read_access" {
  backend                          = var.auth_backend_kubernetes_path
  role_name                        = "${var.team_name}-tools-access-read"
  token_policies                   = [vault_policy.read_access.name]
  token_max_ttl                    = 100
  bound_service_account_names      = var.kubernetes_service_accounts
  bound_service_account_namespaces = var.kubernetes_namespaces
  audience                         = "vault"
}

resource "vault_github_team" "ui_write_access" {
  backend  = var.auth_backend_github_user_path
  team     = var.team_name
  policies = [vault_policy.write_access.name]
}

The variables are as expected and the vault_policy's are also as expected

Debug Output

Not relevant

Panic Output

Stack trace from the terraform-provider-vault_v3.12.0_x5 plugin:

panic: runtime error: invalid memory address or nil pointer dereference

[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x1082397]

goroutine 148 [running]:

[github.com/hashicorp/terraform-provider-vault/vault.githubTeamRead(0xc0005c5780](http://github.com/hashicorp/terraform-provider-vault/vault.githubTeamRead(0xc0005c5780), {0x12eba20?, 0xc0002b60c0?})
[github.com/hashicorp/terraform-provider-vault/vault/resource_github_team.go:101](http://github.com/hashicorp/terraform-provider-vault/vault/resource_github_team.go:101) +0x137
[github.com/hashicorp/terraform-provider-vault/vault.ReadWrapper.func1(0x7f7ac3cb4518](http://github.com/hashicorp/terraform-provider-vault/vault.ReadWrapper.func1(0x7f7ac3cb4518)?, {0x12eba20, 0xc0002b60c0})
[github.com/hashicorp/terraform-provider-vault/vault/provider.go:835](http://github.com/hashicorp/terraform-provider-vault/vault/provider.go:835) +0x5a
[github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x181d2d0](http://github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x181d2d0)?, {0x181d2d0?, 0xc0009505d0?}, 0xd?, {0x12eba20?, 0xc0002b60c0?})
[github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:712](http://github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:712) +0x178
[github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc000a51dc0](http://github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc000a51dc0), {0x181d2d0, 0xc0009505d0}, 0xc000fa8f70, {0x12eba20, 0xc0002b60c0})
[github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:1015](http://github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/resource.go:1015) +0x585
[github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0007bc600](http://github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0007bc600), {0x181d228?, 0xc000cbc600?}, 0xc000cbc680)
[github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/grpc_provider.go:613](http://github.com/hashicorp/terraform-plugin-sdk/v2@v2.16.0/helper/schema/grpc_provider.go:613) +0x4a5
[github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc000c15040](http://github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc000c15040), {0x181d2d0?, 0xc000950030?}, 0xc000f708a0)
[github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/tf5server/server.go:746](http://github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/tf5server/server.go:746) +0x43d
[github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x13cbea0](http://github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler(%7B0x13cbea0)?, 0xc000c15040}, {0x181d2d0, 0xc000950030}, 0xc000f40460, 0x0)
[github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349](http://github.com/hashicorp/terraform-plugin-go@v0.9.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:349) +0x170
[google.golang.org/grpc.(*Server).processUnaryRPC(0xc000a2c5a0](http://google.golang.org/grpc.(*Server).processUnaryRPC(0xc000a2c5a0), {0x1822120, 0xc0000d7040}, 0xc00053afc0, 0xc0006e5e90, 0x2104ff0, 0x0)
[google.golang.org/grpc@v1.50.0/server.go:1318](http://google.golang.org/grpc@v1.50.0/server.go:1318) +0xb2b
[google.golang.org/grpc.(*Server).handleStream(0xc000a2c5a0](http://google.golang.org/grpc.(*Server).handleStream(0xc000a2c5a0), {0x1822120, 0xc0000d7040}, 0xc00053afc0, 0x0)
[google.golang.org/grpc@v1.50.0/server.go:1659](http://google.golang.org/grpc@v1.50.0/server.go:1659) +0xa2f
[google.golang.org/grpc.(*Server).serveStreams.func1.2()](http://google.golang.org/grpc.(*Server).serveStreams.func1.2())
[google.golang.org/grpc@v1.50.0/server.go:955](http://google.golang.org/grpc@v1.50.0/server.go:955) +0x98
created by [google.golang.org/grpc.(*Server).serveStreams.func1](http://google.golang.org/grpc.(*Server).serveStreams.func1)
[google.golang.org/grpc@v1.50.0/server.go:953](http://google.golang.org/grpc@v1.50.0/server.go:953) +0x28a

Error: The terraform-provider-vault_v3.12.0_x5 plugin crashed!

Expected Behavior

The auth method that underlies the role was deleted. So in this case both auth/github/map/teams/atat and auth/kubernetes/role/atat-tools-access-read. The underlying authentication methods (github and kubernetes) were renamed by a different terraform pipeline with a different terraform state. I would have expected that Terraform would recognize that the configuration changed outside of the bounds of this state.

Actual Behavior

Error: Plugin did not respond

with module.team-access.vault_kubernetes_auth_backend_role.tools_read_access,
on .terraform/modules/team-access/vault-access/[main.tf](http://main.tf/) line 51, in resource "vault_kubernetes_auth_backend_role" "tools_read_access":
51: resource "vault_kubernetes_auth_backend_role" "tools_read_access" {

The plugin encountered an error, and failed to respond to the
plugin.(*GRPCProvider).ValidateResourceConfig call. The plugin logs may
contain more details.

Plus a similar message for kubernetes, if that is triggered.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. create a github or kubernetes auth method manually
  2. create a github team or kubernetes role using terraform
  3. delete the auth method manually
  4. run a terraform plan and it will seg fault I tested this on both the github and kubernetes auth method, though seems like it might apply to others as well

Important Factoids

This is executed from a github actions pipeline against a Vault instance hosted in EKS. Should be a pretty typical setup.

This should be a pretty edge case condition that occured trying to change our naming scheme on the fly in dev. Very easy to fix manually (just remove the offending resources from tfstate)

References

N/A

makp0 commented 1 year ago 
resource "vault_auth_backend" "github" {
  type = "github"
}

resource "vault_github_auth_backend" "github" {
  organization = "<sensititive>"
}

resource "vault_policy" "dev" {
  name   = "dev"
  policy = <<EOT
path "*" {
  capabilities = ["read"]
}
EOT
}

resource "vault_github_team" "dev" {
  backend = vault_auth_backend.github.path
  team = "dev"
  policies = [vault_policy.dev.name]
}

After applying, I went to Vault UI and deleted /github auth method this started happening

Stack trace from the terraform-provider-vault_v3.21.0_x5 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x11ca0d7]

goroutine 114 [running]:
github.com/hashicorp/terraform-provider-vault/vault.githubTeamRead(0xc000629200, {0x148a680?, 0xc000d02500?})
github.com/hashicorp/terraform-provider-vault/vault/resource_github_team.go:104 +0x137
github.com/hashicorp/terraform-provider-vault/internal/provider.ReadWrapper.func1(0x0?, {0x148a680, 0xc000d02500})
github.com/hashicorp/terraform-provider-vault/internal/provider/provider.go:241 +0x5a
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0x19ce8e8?, {0x19ce8e8?, 0xc000b37b90?}, 0xd?, {0x148a680?, 0xc000d02500?})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:783 +0x178
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0004dd340, {0x19ce8e8, 0xc000b37b90}, 0xc000b7dc70, {0x148a680, 0xc000d02500})
github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:1089 +0x59e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc0008e4948, {0x19ce8e8?, 0xc000b37a70?}, 0xc000b46200)
github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/grpc_provider.go:649 +0x4a5
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc0002d00a0, {0x19ce8e8?, 0xc000b372c0?}, 0xc000a51a40)
github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/tf5server/server.go:789 +0x4b1
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0x156c320?, 0xc0002d00a0}, {0x19ce8e8, 0xc000b372c0}, 0xc000621b90, 0x0)
github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:431 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00015a3c0, {0x19d4260, 0xc000a54340}, 0xc000b3ed80, 0xc0006833e0, 0x23cf348, 0x0)
google.golang.org/grpc@v1.58.2/server.go:1376 +0xdd2
google.golang.org/grpc.(*Server).handleStream(0xc00015a3c0, {0x19d4260, 0xc000a54340}, 0xc000b3ed80, 0x0)
google.golang.org/grpc@v1.58.2/server.go:1753 +0xa36
google.golang.org/grpc.(*Server).serveStreams.func1.1()
google.golang.org/grpc@v1.58.2/server.go:998 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/grpc@v1.58.2/server.go:996 +0x18c

Error: The terraform-provider-vault_v3.21.0_x5 plugin crashed!
dcaputo-harmoni commented 5 months ago

I am seeing this same error with the same fact pattern as above (renaming a github auth method) - results in a situation where the plugin crashes on every terraform apply.

Just FYI, the workaround I came up with was to manually delete the resources using terraform rm vault_github_auth_backend.<your backend name> and terraform rm vault_github_team.<your team name> (and deleting the corresponding vault auth resources) to start over.