panic: runtime error: invalid memory address or nil pointer dereference

[ture-karlsson]

Hello!

I get this crash when running a simple test with the vault provider. It does not appear on first plan/apply, but after I have applied and destroyed once and then try to apply again: https://gist.github.com/ture-karlsson/ea17eedf3ade9f6fbc0248059f013316

Terraform Version

$ terraform --version
Terraform v0.14.10
+ provider registry.terraform.io/hashicorp/vault v2.19.0

Terraform Configuration Files

Just a simple demo:

providers.tf

provider "vault" {
  address               = var.vault_addr
  namespace             = var.vault_namespace
  max_lease_ttl_seconds = 24000

  auth_login {
    path      = "auth/approle/login"
    namespace = var.vault_namespace
    parameters = {
      role_id   = var.login_approle_role_id
      secret_id = var.login_approle_secret_id
    }
  }
}

main.tf

data "vault_generic_secret" "tures_secret" {
  path = "secret/ture"
}

Panic Output

https://gist.github.com/ture-karlsson/506b9ec9028f195ec699f9a6e44921b9

Expected Behavior

Apply would be OK second time as well

Actual Behavior

Terraform crash above

Steps to Reproduce

1. terraform apply
2. terraform destroy
3. terraform apply

[ture-karlsson]

2021-04-12T10:56:22.139+0200 [DEBUG] plugin.terraform-provider-vault_v2.19.0_x4: panic: runtime error: invalid memory address or nil pointer dereference
2021-04-12T10:56:22.139+0200 [DEBUG] plugin.terraform-provider-vault_v2.19.0_x4: [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1b4a8d1]
2021-04-12T10:56:22.139+0200 [DEBUG] plugin.terraform-provider-vault_v2.19.0_x4: 
2021-04-12T10:56:22.139+0200 [DEBUG] plugin.terraform-provider-vault_v2.19.0_x4: goroutine 14 [running]:
2021-04-12T10:56:22.139+0200 [DEBUG] plugin.terraform-provider-vault_v2.19.0_x4: github.com/hashicorp/terraform-provider-vault/vault.providerConfigure(0xc0002aefc0, 0x0, 0xc000518e00, 0xc0002aefc0, 0x0)
2021-04-12T10:56:22.139+0200 [DEBUG] plugin.terraform-provider-vault_v2.19.0_x4:    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-vault/vault/provider.go:820 +0xdc1

Seems to be related to the client token?

https://github.com/hashicorp/terraform-provider-vault/blob/master/vault/provider.go#L820

[ture-karlsson]

Can anyone have a look at this? Still experience this on Terraform 1.0.0 and Vault provider 2.21.0:

│ Error: Plugin did not respond
│ 
│   with provider["registry.terraform.io/hashicorp/vault"],
│   on main.tf line 14, in provider "vault":
│   14: provider "vault" {
│ 
│ The plugin encountered an error, and failed to respond to the
│ plugin.(*GRPCProvider).ConfigureProvider call. The plugin logs may contain
│ more details.
╵

Stack trace from the terraform-provider-vault_v2.21.0_x4 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xe5603e]

goroutine 10 [running]:
github.com/hashicorp/terraform-provider-vault/vault.providerConfigure(0xc0005e2230, 0x0, 0xc000a60480, 0xc0005e2230, 0x0)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vault/provider.go:836 +0xe9e
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).Configure(0xc000112100, 0xc000152d50, 0x1052cc0, 0xc000152b70)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/provider.go:275 +0xfc
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).Configure(0xc00010a5a0, 0x1387890, 0xc0005e0390, 0xc000246680, 0xc00010a5a0, 0xc0005e0390, 0xc000703ba0)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin/grpc_provider.go:487 +0x2ee
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_Configure_Handler(0x1154300, 0xc00010a5a0, 0x1387890, 0xc0005e0390, 0xc000281320, 0x0, 0x1387890, 0xc0005e0390, 0xc00014c780, 0x1c8)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5/tfplugin5.pb.go:3251 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000676000, 0x1392658, 0xc000116c00, 0xc000248500, 0xc00057e6f0, 0x19cce78, 0x0, 0x0, 0x0)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/google.golang.org/grpc/server.go:1082 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc000676000, 0x1392658, 0xc000116c00, 0xc000248500, 0x0)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/google.golang.org/grpc/server.go:1405 +0xccf
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000a84a0, 0xc000676000, 0x1392658, 0xc000116c00, 0xc000248500)
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/google.golang.org/grpc/server.go:746 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
    /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/hashicorp/terraform-provider-vault/vendor/google.golang.org/grpc/server.go:744 +0xa5

Error: The terraform-provider-vault_v2.21.0_x4 plugin crashed!

[julianxhokaxhiu]

This is superweird because i was running the very same configuration until 2 hours ago and now, all of a sudden, I'm having the very same crash for no apparent reason. I'll continue to investigate in the meantime, but any help is appreciated.

//EDIT: I wiped the plugin cache and redownloaded for like 10 times, until it worked again. Apparently the provider plugin code was downloaded corrupted. Isn't there a sanity check done by TF when downloading plugins?

//EDIT2: I spoke too early. It was a lucky shot, now it's crashing again with the very same error as stated above.

//EDIT3: Found the core issue! At least for me: the user i was using did reach the maximum number of leases available in Vault. I had to revoke them using the relative command and after that it started to work. The way I found out about this was exporting

$ export TF_LOG="DEBUG"
$ export TF_LOG_FILE="tf.log"

and analyzing my own log file understanding what Vault was returning at the POST response section. Would be nice if the provider, instead of crashing brutally and misleading the user, would return the POST error message instead.