search

mrparkers / terraform-provider-keycloak

Terraform provider for Keycloak
https://registry.terraform.io/providers/mrparkers/keycloak/latest/docs
MIT License
607 stars 295 forks source link

Keycloak V21.1.1: Issue with `keycloak_authentication_execution_config` #839

Closed KlausNie closed 1 year ago

KlausNie commented 1 year ago

I know that this provider does not yet support keycloak with version 21.1.1 but only 21.0.1.

This is something to be aware when supporting that version (21.1.1).

We migrated from 20.0.5 to 21.1.1. Then some terraform apply started to break. Specifically, those which were touching keycloak_authentication_execution_config.

This is what we are executing

resource "keycloak_authentication_execution" "execution_restrict_client_auth_authenticator" {
  realm_id          = var.realm_id
  parent_flow_alias = keycloak_authentication_subflow.subflow_browser_forms.alias # parent flow
  authenticator     = "restrict-client-auth-authenticator"
  requirement       = "REQUIRED"

  depends_on = [keycloak_authentication_execution.execution_auth_username_password_form] # previous authenticator
}

resource "keycloak_authentication_execution_config" "execution_restrict_client_auth_authenticator_config" {
  realm_id     = var.realm_id
  execution_id = keycloak_authentication_execution.execution_restrict_client_auth_authenticator.id # authenticator id
  alias        = "restricted-access"
  config = {}
}

It seems like, for some reason, when executing these terraform statements, keycloak does not create valid entries for AUTHENTICATOR_CONFIG in the database (we are using MySQL).

image

The ID field seems to be an empty string, which then breaks once it tries to update/create a new one for the same authenticator config.

Yes, we are using custom providers for this and that could also be the root cause for this error. Note that it was working with previous versions of Keycloak + the provider.

The last few lines of the terraform trace:

2023-05-08T10:11:07.677+0200 [TRACE] vertex "root": starting visit (terraform.graphNodeRoot)
2023-05-08T10:11:07.677+0200 [TRACE] vertex "root": visit complete
2023-05-08T10:11:07.677+0200 [TRACE] vertex "keycloak_realm.realm (expand)": dynamic subgraph completed successfully
2023-05-08T10:11:07.677+0200 [TRACE] vertex "keycloak_realm.realm (expand)": visit complete
2023-05-08T10:11:07.677+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id (expand)": starting visit (*terraform.nodeExpandModuleVariable)
2023-05-08T10:11:07.677+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id (expand)": expanding dynamic subgraph
2023-05-08T10:11:07.677+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.keycloak_authentication_execution.execution_restrict_client_auth_authenticator (expand)": visit complete
2023-05-08T10:11:07.677+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id (expand)": entering dynamic subgraph
2023-05-08T10:11:07.677+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id": starting visit (*terraform.nodeModuleVariable)
2023-05-08T10:11:07.677+0200 [TRACE] nodeModuleVariable: evaluating module.restricted-authentication-browser-flow.var.realm_id
2023-05-08T10:11:07.679+0200 [TRACE] prepareFinalInputVariableValue: preparing module.restricted-authentication-browser-flow.var.realm_id
2023-05-08T10:11:07.679+0200 [TRACE] BuiltinEvalContext: Storing final value for variable module.restricted-authentication-browser-flow.var.realm_id
2023-05-08T10:11:07.679+0200 [TRACE] evalVariableValidations: no validation rules declared for module.restricted-authentication-browser-flow.var.realm_id, so skipping
2023-05-08T10:11:07.679+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id": visit complete
2023-05-08T10:11:07.679+0200 [TRACE] vertex "root": starting visit (terraform.graphNodeRoot)
2023-05-08T10:11:07.679+0200 [TRACE] vertex "root": visit complete
2023-05-08T10:11:07.679+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id (expand)": dynamic subgraph completed successfully
2023-05-08T10:11:07.679+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.var.realm_id (expand)": visit complete
2023-05-08T10:11:07.679+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config": starting visit (*terraform.NodeApplyableResourceInstance)
2023-05-08T10:11:07.679+0200 [TRACE] readDiff: Read Create change from plan for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.679+0200 [TRACE] readResourceInstanceState: reading state for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.679+0200 [TRACE] readResourceInstanceState: no state present for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.679+0200 [TRACE] readDiff: Read Create change from plan for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.679+0200 [TRACE] Re-validating config for "module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config"
2023-05-08T10:11:07.679+0200 [TRACE] GRPCProvider: ValidateResourceConfig
2023-05-08T10:11:07.680+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Received request: tf_proto_version=5.3 tf_provider_addr=provider tf_req_id=e1da22f6-03b4-7dca-ad7a-5552ff67c974 @module=sdk.proto tf_resource_type=keycloak_authentication_execution_config tf_rpc=ValidateResourceTypeConfig @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:679 timestamp=2023-05-08T10:11:07.680+0200
2023-05-08T10:11:07.680+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Sending request downstream: tf_rpc=ValidateResourceTypeConfig tf_proto_version=5.3 tf_req_id=e1da22f6-03b4-7dca-ad7a-5552ff67c974 tf_resource_type=keycloak_authentication_execution_config tf_provider_addr=provider @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:17 @module=sdk.proto timestamp=2023-05-08T10:11:07.680+0200
2023-05-08T10:11:07.680+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Calling downstream: tf_rpc=ValidateResourceTypeConfig @module=sdk.helper_schema tf_provider_addr=provider tf_req_id=e1da22f6-03b4-7dca-ad7a-5552ff67c974 tf_resource_type=keycloak_authentication_execution_config @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.23.0/helper/schema/grpc_provider.go:245 timestamp=2023-05-08T10:11:07.680+0200
2023-05-08T10:11:07.680+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Called downstream: @module=sdk.helper_schema tf_provider_addr=provider tf_req_id=e1da22f6-03b4-7dca-ad7a-5552ff67c974 tf_resource_type=keycloak_authentication_execution_config @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.23.0/helper/schema/grpc_provider.go:247 tf_rpc=ValidateResourceTypeConfig timestamp=2023-05-08T10:11:07.680+0200
2023-05-08T10:11:07.680+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Received downstream response: tf_rpc=ValidateResourceTypeConfig @module=sdk.proto diagnostic_warning_count=0 tf_proto_version=5.3 tf_req_duration_ms=0 tf_req_id=e1da22f6-03b4-7dca-ad7a-5552ff67c974 tf_resource_type=keycloak_authentication_execution_config @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:37 diagnostic_error_count=0 tf_provider_addr=provider timestamp=2023-05-08T10:11:07.680+0200
2023-05-08T10:11:07.680+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Served request: tf_rpc=ValidateResourceTypeConfig @module=sdk.proto tf_resource_type=keycloak_authentication_execution_config tf_proto_version=5.3 tf_provider_addr=provider tf_req_id=e1da22f6-03b4-7dca-ad7a-5552ff67c974 @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:699 timestamp=2023-05-08T10:11:07.680+0200
2023-05-08T10:11:07.680+0200 [TRACE] GRPCProvider: PlanResourceChange
2023-05-08T10:11:07.681+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Received request: @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:770 tf_req_id=58702866-45ff-7284-87d7-2eb6f5eba48d tf_resource_type=keycloak_authentication_execution_config tf_rpc=PlanResourceChange @module=sdk.proto tf_proto_version=5.3 tf_provider_addr=provider timestamp=2023-05-08T10:11:07.681+0200
2023-05-08T10:11:07.681+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Sending request downstream: @module=sdk.proto tf_rpc=PlanResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:17 tf_proto_version=5.3 tf_provider_addr=provider tf_req_id=58702866-45ff-7284-87d7-2eb6f5eba48d tf_resource_type=keycloak_authentication_execution_config timestamp=2023-05-08T10:11:07.681+0200
2023-05-08T10:11:07.681+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Received downstream response: diagnostic_error_count=0 tf_proto_version=5.3 tf_req_duration_ms=0 tf_req_id=58702866-45ff-7284-87d7-2eb6f5eba48d tf_resource_type=keycloak_authentication_execution_config @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:37 @module=sdk.proto diagnostic_warning_count=0 tf_provider_addr=provider tf_rpc=PlanResourceChange timestamp=2023-05-08T10:11:07.681+0200
2023-05-08T10:11:07.681+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Served request: tf_proto_version=5.3 tf_provider_addr=provider @module=sdk.proto tf_req_id=58702866-45ff-7284-87d7-2eb6f5eba48d tf_resource_type=keycloak_authentication_execution_config tf_rpc=PlanResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:796 timestamp=2023-05-08T10:11:07.681+0200
2023-05-08T10:11:07.681+0200 [WARN]  Provider "registry.terraform.io/mrparkers/keycloak" produced an invalid plan for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .config: planned value cty.NullVal(cty.Map(cty.String)) does not match config value cty.MapValEmpty(cty.String)
2023-05-08T10:11:07.681+0200 [TRACE] checkPlannedChange: Verifying that actual change (action Create) matches planned change (action Create)
module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config: Creating...
2023-05-08T10:11:07.682+0200 [INFO]  Starting apply for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.682+0200 [DEBUG] module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config: applying the planned Create change
2023-05-08T10:11:07.682+0200 [TRACE] GRPCProvider: ApplyResourceChange
2023-05-08T10:11:07.682+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Received request: @module=sdk.proto tf_proto_version=5.3 tf_provider_addr=provider tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_resource_type=keycloak_authentication_execution_config tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:805 timestamp=2023-05-08T10:11:07.682+0200
2023-05-08T10:11:07.682+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Sending request downstream: tf_proto_version=5.3 @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:17 tf_provider_addr=provider tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_resource_type=keycloak_authentication_execution_config tf_rpc=ApplyResourceChange @module=sdk.proto timestamp=2023-05-08T10:11:07.682+0200
2023-05-08T10:11:07.682+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Calling downstream: tf_provider_addr=provider tf_resource_type=keycloak_authentication_execution_config tf_rpc=ApplyResourceChange @module=sdk.helper_schema tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.23.0/helper/schema/resource.go:836 timestamp=2023-05-08T10:11:07.682+0200
2023-05-08T10:11:07.682+0200 [DEBUG] provider.terraform-provider-keycloak_v4.2.0: Sending request: @module=provider body={"id":"","alias":"restricted-access","config":{}} method=POST tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_resource_type=keycloak_authentication_execution_config tf_rpc=ApplyResourceChange @caller=github.com/mrparkers/terraform-provider-keycloak/keycloak/keycloak_client.go:313 path=//admin/realms/bhz-realm/authentication/executions/ac535d04-41cf-4e33-8df1-7b3e777b1e87/config tf_provider_addr=provider timestamp=2023-05-08T10:11:07.682+0200
2023-05-08T10:11:07.740+0200 [DEBUG] provider.terraform-provider-keycloak_v4.2.0: Received response: tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_resource_type=keycloak_authentication_execution_config @caller=github.com/mrparkers/terraform-provider-keycloak/keycloak/keycloak_client.go:360 @module=provider body={"error":"unknown_error"} status="409 Conflict" tf_provider_addr=provider tf_rpc=ApplyResourceChange timestamp=2023-05-08T10:11:07.740+0200
2023-05-08T10:11:07.740+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Called downstream: @module=sdk.helper_schema tf_provider_addr=provider tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_resource_type=keycloak_authentication_execution_config tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.23.0/helper/schema/resource.go:838 timestamp=2023-05-08T10:11:07.740+0200
2023-05-08T10:11:07.740+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Received downstream response: @module=sdk.proto diagnostic_error_count=1 diagnostic_warning_count=0 tf_provider_addr=provider tf_req_duration_ms=58 tf_resource_type=keycloak_authentication_execution_config tf_proto_version=5.3 tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:37 timestamp=2023-05-08T10:11:07.740+0200
2023-05-08T10:11:07.740+0200 [ERROR] provider.terraform-provider-keycloak_v4.2.0: Response contains error diagnostic: tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_severity=ERROR diagnostic_summary="error sending POST request to //admin/realms/bhz-realm/authentication/executions/ac535d04-41cf-4e33-8df1-7b3e777b1e87/config: 409 Conflict. Response body: {"error":"unknown_error"}" tf_proto_version=5.3 tf_provider_addr=provider tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 diagnostic_detail= tf_resource_type=keycloak_authentication_execution_config timestamp=2023-05-08T10:11:07.740+0200
2023-05-08T10:11:07.741+0200 [TRACE] provider.terraform-provider-keycloak_v4.2.0: Served request: @caller=github.com/hashicorp/terraform-plugin-go@v0.14.0/tfprotov5/tf5server/server.go:831 tf_provider_addr=provider tf_req_id=d54d3312-c920-4a77-8b33-1025b1d8e1d4 tf_resource_type=keycloak_authentication_execution_config tf_rpc=ApplyResourceChange @module=sdk.proto tf_proto_version=5.3 timestamp=2023-05-08T10:11:07.740+0200
2023-05-08T10:11:07.741+0200 [TRACE] maybeTainted: module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config encountered an error during creation, so it is now marked as tainted
2023-05-08T10:11:07.741+0200 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.741+0200 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: removing state object for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.741+0200 [TRACE] evalApplyProvisioners: module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config is tainted, so skipping provisioning
2023-05-08T10:11:07.741+0200 [TRACE] maybeTainted: module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config was already tainted, so nothing to do
2023-05-08T10:11:07.741+0200 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.741+0200 [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: removing state object for module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config
2023-05-08T10:11:07.742+0200 [ERROR] vertex "module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config" error: error sending POST request to //admin/realms/bhz-realm/authentication/executions/ac535d04-41cf-4e33-8df1-7b3e777b1e87/config: 409 Conflict. Response body: {"error":"unknown_error"}
2023-05-08T10:11:07.742+0200 [TRACE] vertex "module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config": visit complete, with errors
2023-05-08T10:11:07.742+0200 [TRACE] dag/walk: upstream of "module.restricted-authentication-browser-flow (close)" errored, so skipping
2023-05-08T10:11:07.742+0200 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/mrparkers/keycloak\"] (close)" errored, so skipping
2023-05-08T10:11:07.742+0200 [TRACE] dag/walk: upstream of "root" errored, so skipping
2023-05-08T10:11:07.742+0200 [DEBUG] states/remote: state read serial is: 404; serial is: 404
2023-05-08T10:11:07.742+0200 [DEBUG] states/remote: state read lineage is: d6affe8e-e63c-a1b3-825c-f4f3102bb4c2; lineage is: d6affe8e-e63c-a1b3-825c-f4f3102bb4c2
╷
│ Error: error sending POST request to //admin/realms/bhz-realm/authentication/executions/ac535d04-41cf-4e33-8df1-7b3e777b1e87/config: 409 Conflict. Response body: {"error":"unknown_error"}
│ 
│   with module.restricted-authentication-browser-flow.keycloak_authentication_execution_config.execution_restrict_client_auth_authenticator_config,
│   on ../modules/authentication/authentication.tf line 65, in resource "keycloak_authentication_execution_config" "execution_restrict_client_auth_authenticator_config":
│   65: resource "keycloak_authentication_execution_config" "execution_restrict_client_auth_authenticator_config" {
│ 
╵
2023-05-08T10:11:07.750+0200 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2023-05-08T10:11:07.751+0200 [DEBUG] provider: plugin process exited: path=.terraform/providers/registry.terraform.io/mrparkers/keycloak/4.2.0/darwin_arm64/terraform-provider-keycloak_v4.2.0 pid=91805
2023-05-08T10:11:07.751+0200 [DEBUG] provider: plugin exited
kherock commented 1 year ago

Also seeing this issue, but it occurs on configurations that are for native keycloak providers as well.

kherock commented 1 year ago

Looking into this a bit further -

It looks like the ID keycloak generates is indeed an empty string (I'm using Postgres 13). The config is actually there and is linked by this empty string "" ID, but terraform-provider-keycloak just assumes that the resource wasn't created. The very first apply actually creates this error:

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to keycloak_authentication_execution_config.post_login_session_limiter_config, provider "provider[\"registry.terraform.io/mrparkers/keycloak\"]" produced an unexpected new
│ value: Root resource was present, but now absent.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker

Subsequent applies generate a 409 conflict due to the empty string ID already in use.

This empty ID is not created for resources created by the admin UI. The problem also isn't timing related as the empty ID still manifests when I create a Terraform plan that creates the config on an existing execution.

kherock commented 1 year ago

Here's where the regression occurred! https://github.com/keycloak/keycloak/pull/19466

Keycloak 21.1 now supports preserving execution config IDs during import. However, it doesn't include basic validation of the imported IDs (empty IDs are allowed). The representation sent by this provider currently includes an empty string "id": "" in its JSON payload.

I'm not familiar with Go, but it looks like the fix might be to apply an omitempty decorator to Id field in the struct definition?

https://github.com/mrparkers/terraform-provider-keycloak/blob/78f6b6e3375a06ff6c45a86699065b8a27b8539d/keycloak/authentication_execution_config.go#L9-L15

KlausNie commented 1 year ago

@kherock awesome! That could really be all that is needed!