Closed alordthorsen closed 2 years ago
2022-06-22T23:50:40.484Z [WARN] Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.replicate_oltp.aws_dms_endpoint.this["postgresql_source"], but we are tolerating it because it is using the legacy plugin SDK.
The following problems may be the cause of any confusing errors from downstream operations:
- .tags: planned value cty.NullVal(cty.Map(cty.String)) does not match config value cty.MapValEmpty(cty.String)
- .redshift_settings: attribute representing nested block must not be unknown itself; set nested attribute values to unknown instead
2022-06-22T23:50:40.485Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState to workingState for module.replicate_oltp.aws_dms_endpoint.this["postgresql_source"]
2022-06-22T23:50:40.485Z [TRACE] NodeAbstractResouceInstance.writeResourceInstanceState: writing state object for module.replicate_oltp.aws_dms_endpoint.this["postgresql_source"]
2022-06-22T23:50:40.485Z [ERROR] vertex "module.replicate_oltp.aws_dms_endpoint.this[\"rates_kinesis_target\"]" error: 1 error occurred:
* kinesis_settings must be set when engine_name = "kinesis"
2022-06-22T23:50:40.485Z [TRACE] vertex "module.replicate_oltp.aws_dms_endpoint.this[\"rates_kinesis_target\"]": visit complete, with errors
2022-06-22T23:50:40.485Z [TRACE] writeChange: recorded Create change for module.replicate_oltp.aws_dms_endpoint.this["postgresql_source"]
2022-06-22T23:50:40.486Z [TRACE] vertex "module.replicate_oltp.aws_dms_endpoint.this[\"postgresql_source\"]": visit complete
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "root" errored, so skipping
2022-06-22T23:50:40.485Z [TRACE] provider.terraform-provider-aws_v4.19.0_x5: Served request: @caller=github.com/hashicorp/terraform-plugin-go@v0.9.1/tfprotov5/tf5server/server.go:791 @module=sdk.proto tf_proto_version=5.2 tf_provider_addr=provider tf_resource_type=aws_dms_endpoint tf_req_id=4fa6f44f-c38e-fbf5-37af-3b50cbc67640 tf_rpc=PlanResourceChange timesta
mp=2022-06-22T23:50:40.485Z
2022-06-22T23:50:40.486Z [TRACE] vertex "module.replicate_oltp.aws_dms_endpoint.this": dynamic subgraph encountered errors: 1 error occurred:
* kinesis_settings must be set when engine_name = "kinesis"
2022-06-22T23:50:40.486Z [ERROR] vertex "module.replicate_oltp.aws_dms_endpoint.this" error: 1 error occurred:
* kinesis_settings must be set when engine_name = "kinesis"
2022-06-22T23:50:40.486Z [TRACE] vertex "module.replicate_oltp.aws_dms_endpoint.this": visit complete, with errors
2022-06-22T23:50:40.486Z [TRACE] vertex "module.replicate_oltp.aws_dms_endpoint.this (expand)": dynamic subgraph encountered errors: 1 error occurred:
* kinesis_settings must be set when engine_name = "kinesis"
2022-06-22T23:50:40.486Z [ERROR] vertex "module.replicate_oltp.aws_dms_endpoint.this (expand)" error: 1 error occurred:
* kinesis_settings must be set when engine_name = "kinesis"
2022-06-22T23:50:40.486Z [TRACE] vertex "module.replicate_oltp.aws_dms_endpoint.this (expand)": visit complete, with errors
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "module.replicate_oltp.output.endpoints (expand)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "module.replicate_oltp.aws_dms_replication_task.this (expand)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "module.replicate_oltp.output.replication_tasks (expand)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "module.replicate_oltp.aws_dms_event_subscription.this (expand)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "module.replicate_oltp.output.event_subscriptions (expand)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "module.replicate_oltp (close)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/aws\"] (close)" errored, so skipping
2022-06-22T23:50:40.486Z [TRACE] dag/walk: upstream of "root" errored, so skipping
2022-06-22T23:50:40.486Z [INFO] backend/local: plan operation completed
╷
│ Error: 1 error occurred:
│ * kinesis_settings must be set when engine_name = "kinesis"
│
│
│
│ with module.replicate_oltp.aws_dms_endpoint.this["rates_kinesis_target"],
│ on .terraform/modules/replicate_oltp/main.tf line 160, in resource "aws_dms_endpoint" "this":
│ 160: resource "aws_dms_endpoint" "this" {
I setup an S3 target just to test this behavior and am running into the same error with the new target
s3_target = {
endpoint_id = "${local.name}-s3-target"
endpoint_type = "target"
engine_name = "s3"
ssl_mode = "none"
s3_settings = {
bucket_folder = "${var.env_name}-psql-raw-cdc"
bucket_name = module.replication_backup.bucket_name # to avoid https://github.com/hashicorp/terraform/issues/4149
service_access_role_arn = aws_iam_role.dms_write_to_s3_target.arn
data_format = "parquet"
EnableStatistics = true,
ParquetTimestampInMillisecond = true,
DatePartitionEnabled = false
}
}
same error different target
│ Error: 1 error occurred:
│ * s3_settings must be set when engine_name = "s3"
│
│
│
│ with module.replicate_oltp.aws_dms_endpoint.this["s3_target"],
│ on .terraform/modules/replicate_oltp/main.tf line 160, in resource "aws_dms_endpoint" "this":
│ 160: resource "aws_dms_endpoint" "this" {
After some further digging this settings declaration gets terraform plan to work
kinesis_settings = {
# service_access_role_arn = aws_iam_role.dms_write_to_rating_data_sync.arn
stream_arn = data.aws_kinesis_stream.rating_replication_stream.arn
partition_include_schema_table = true
include_partition_value = true
}
Which lead me to https://github.com/hashicorp/terraform/issues/30937
This issue has been resolved in version 1.5.2 :tada:
@alordthorsen can you try using v1.5.2 and let us know if that resolves your issue
Ha, this was fast enough that I don't think terraform.io has updated yet. =P
│ Error: Unresolvable module version constraint
│
│ There is no available version of module "registry.terraform.io/terraform-aws-modules/dms/aws" (dms.tf:9) which matches the given version constraint. The newest available version is 1.5.1.
I'll give it a try when it's able to download.
Cooool, this is working for me. Thank you for all the help.
Can either of you explain to me exactly why we needed to switch from try
to lookup
in this case? Does this indicate that try
is a function to be avoided or is there some interaction with dynamic
that makes try
unreliable?
as @antonbabenko has discovered, functions such as try()
, can()
, jsonencode()
appear to either be eagerly or exhaustively evaluated which causes issues when values within the underlying map are not yet known. lookup()
does not appear to follow this same approach and instead appears to be more of a lazy evaluation - @apparentlymart might know better; I haven't gone in to the source to fully grok the difference in implementations to say definitively
The issue you linked above is well known but typically associated with for_each
situations where the keys of the map that are being iterated over is not static and needs to be evaluated which leads to the common error The "for_each" value depends on resource attributes that cannot be determined until apply
which is not the case here
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.
Description
I'm hitting
I have re-initialized my terraform project after cleaning up the
.terraform
directory.Versions
Module version [Required]: 1.5.1
Terraform version:
Provider version(s):
Providers required by configuration: . ├── provider[registry.terraform.io/shippo/aws] ~> 3.0 ├── provider[registry.terraform.io/hashicorp/random] ├── module.dms_write_to_rating_data_sync_policy │ └── provider[registry.terraform.io/hashicorp/aws] >= 3.35.0 ├── module.postgres_dms_instance_access │ └── provider[registry.terraform.io/hashicorp/aws] >= 3.0.0 ├── module.replicate_oltp │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.6.0 ├── module.replication_backup │ └── provider[registry.terraform.io/shippo/aws] ~> 3.0 └── module.defaults ├── provider[registry.terraform.io/hashicorp/aws] └── provider[terraform.io/builtin/terraform]
endpoints = { rates_kinesis_target = { endpoint_id = "${local.name}-rates-kinesis-target" endpoint_type = "target" engine_name = "kinesis"
Terraform init -backend-config=envs/dev-main/backend.tf Terraform plan