Open emnedre opened 10 months ago
I've run into this a few times and it appears to be centered around defaults Cloud Formation populates.
When you create a bucket using the following definition, the Terraform state contains default ownership_controls
and bucket_encryption
configurations. These controls are not present in a simple awscc_s3_bucket
configuration resulting in a difference between the configuration and state.
Users can workaround this problem by populating the ownership_controls
and bucket_encryption
arguments. The provider should be handling this by default.
Steps to reproduce.
resource "awscc_s3_bucket" "example" {
bucket_name = "example-bucket-958630"
}
Review the state information for resource awscc_s3_bucket.example
after the first apply. Notice the bucket_encryption and ownership_controls are now populated
# awscc_s3_bucket.example:
resource "awscc_s3_bucket" "example" {
arn = "arn:aws:s3:::example-bucket-958630"
bucket_encryption = {
server_side_encryption_configuration = [
{
bucket_key_enabled = false
server_side_encryption_by_default = {
sse_algorithm = "AES256"
}
},
]
}
bucket_name = "example-bucket-958630"
domain_name = "example-bucket-958630.s3.amazonaws.com"
dual_stack_domain_name = "example-bucket-958630.s3.dualstack.us-east-1.amazonaws.com"
id = "example-bucket-958630"
ownership_controls = {
rules = [
{
object_ownership = "BucketOwnerEnforced"
},
]
}
public_access_block_configuration = {
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
regional_domain_name = "example-bucket-958630.s3.us-east-1.amazonaws.com"
website_url = "http://example-bucket-958630.s3-website-us-east-1.amazonaws.com"
}
Compared to the data CloudControl returns in the .ResourceDescription.Properties
field.
{
"PublicAccessBlockConfiguration": {
"RestrictPublicBuckets": true,
"BlockPublicPolicy": true,
"BlockPublicAcls": true,
"IgnorePublicAcls": true
},
"BucketName": "example-bucket-958630",
"RegionalDomainName": "example-bucket-958630.s3.us-east-1.amazonaws.com",
"OwnershipControls": {
"Rules": [
{
"ObjectOwnership": "BucketOwnerEnforced"
}
]
},
"DomainName": "example-bucket-958630.s3.amazonaws.com",
"BucketEncryption": {
"ServerSideEncryptionConfiguration": [
{
"BucketKeyEnabled": false,
"ServerSideEncryptionByDefault": {
"SSEAlgorithm": "AES256"
}
}
]
},
"WebsiteURL": "http://example-bucket-958630.s3-website-us-east-1.amazonaws.com",
"DualStackDomainName": "example-bucket-958630.s3.dualstack.us-east-1.amazonaws.com",
"Arn": "arn:aws:s3:::example-bucket-958630"
}
Populate the ownership_controls
and bucket_encryption
arguments in the HCL to be aligned with CloudControl and the Terraform state file.
resource "awscc_s3_bucket" "example" {
bucket_name = "example-bucket-958630"
ownership_controls = {
rules = [{
object_ownership = "BucketOwnerEnforced"
}]
}
bucket_encryption = {
server_side_encryption_configuration = [{
server_side_encryption_by_default = {
sse_algorithm = "AES256"
}
}]
}
}
TLDR: I would expect the provider to handle the ownership_controls
and default_encryption
defaults and not produce a diff. It's unclear to me where these defaults are being defined since the internal resources are generated based on the CloudFormation schemas.
got bitten by this again today during my test, which reminded me to get back to this issue.
in my case, the trigger was this particular attribute object_lock_enabled
# awscc_s3_bucket.example must be replaced
-/+ resource "awscc_s3_bucket" "example" {
. . .
~ id = "example-cloudtrail-368288356491" -> (known after apply)
. . .
+ object_lock_enabled = (known after apply) # forces replacement
on the Terraform statefile, this attribute was set to null, since I did not declare it on the Terraform config. subsequently, CCAPI also did not return the value at all:
{
"PublicAccessBlockConfiguration": {
"RestrictPublicBuckets": true,
"BlockPublicPolicy": true,
"BlockPublicAcls": true,
"IgnorePublicAcls": true
},
"BucketName": "example-cloudtrail-368288356491",
"RegionalDomainName": "example-cloudtrail-368288356491.s3.us-east-1.amazonaws.com",
"OwnershipControls": {
"Rules": [
{
"ObjectOwnership": "BucketOwnerEnforced"
}
]
},
"DomainName": "example-cloudtrail-368288356491.s3.amazonaws.com",
"BucketEncryption": {
"ServerSideEncryptionConfiguration": [
{
"BucketKeyEnabled": false,
"ServerSideEncryptionByDefault": {
"SSEAlgorithm": "aws:kms",
"KMSMasterKeyID": "arn:aws:kms:us-east-1:368288356491:key/f84fadd5-d0fd-4abb-9c4b-39a3d378b22b"
}
}
]
},
"WebsiteURL": "http://example-cloudtrail-368288356491.s3-website-us-east-1.amazonaws.com",
"DualStackDomainName": "example-cloudtrail-368288356491.s3.dualstack.us-east-1.amazonaws.com",
"Arn": "arn:aws:s3:::example-cloudtrail-368288356491"
}
This is known issue, related to #1139
however the lack of default values in the Cfn schema still can trigger drift (without replacement) as shown in this issue.
Upon further debug by setting the env var TF_LOG_SDK_PROTO_DATA_DIR
to local directory:
PlanResourceChange_Request_PriorState matches what is on the state file:
{
"accelerate_configuration": null,
"access_control": null,
"analytics_configurations": null,
"arn": "arn:aws:s3:::exmaple-bucket-123123123123",
"bucket_encryption": {
"server_side_encryption_configuration": [
{
"bucket_key_enabled": false,
"server_side_encryption_by_default": {
"kms_master_key_id": null,
"sse_algorithm": "AES256"
}
}
]
},
"bucket_name": "exmaple-bucket-123123123123",
"cors_configuration": null,
"domain_name": "exmaple-bucket-123123123123.s3.amazonaws.com",
"dual_stack_domain_name": "exmaple-bucket-123123123123.s3.dualstack.us-east-1.amazonaws.com",
"id": "exmaple-bucket-123123123123",
"intelligent_tiering_configurations": null,
"inventory_configurations": null,
"lifecycle_configuration": null,
"logging_configuration": null,
"metrics_configurations": null,
"notification_configuration": null,
"object_lock_configuration": null,
"object_lock_enabled": null,
"ownership_controls": {
"rules": [
{
"object_ownership": "BucketOwnerEnforced"
}
]
},
"public_access_block_configuration": {
"block_public_acls": true,
"block_public_policy": true,
"ignore_public_acls": true,
"restrict_public_buckets": true
},
"regional_domain_name": "exmaple-bucket-123123123123.s3.us-east-1.amazonaws.com",
"replication_configuration": null,
"tags": null,
"versioning_configuration": null,
"website_configuration": null,
"website_url": "http://exmaple-bucket-123123123123.s3-website-us-east-1.amazonaws.com"
}
The PlanResourceChange_Request_ProposedNewState set both bucket_encryption
and ownership_controls
to null.
{
"accelerate_configuration": null,
"access_control": null,
"analytics_configurations": null,
"arn": "arn:aws:s3:::exmaple-bucket-123123123123",
"bucket_encryption": null,
"bucket_name": "exmaple-bucket-123123123123",
"cors_configuration": null,
"domain_name": "exmaple-bucket-123123123123.s3.amazonaws.com",
"dual_stack_domain_name": "exmaple-bucket-123123123123.s3.dualstack.us-east-1.amazonaws.com",
"id": "exmaple-bucket-123123123123",
"intelligent_tiering_configurations": null,
"inventory_configurations": null,
"lifecycle_configuration": null,
"logging_configuration": null,
"metrics_configurations": null,
"notification_configuration": null,
"object_lock_configuration": null,
"object_lock_enabled": null,
"ownership_controls": null,
"public_access_block_configuration": {
"block_public_acls": true,
"block_public_policy": true,
"ignore_public_acls": true,
"restrict_public_buckets": true
},
"regional_domain_name": "exmaple-bucket-123123123123.s3.us-east-1.amazonaws.com",
"replication_configuration": null,
"tags": null,
"versioning_configuration": null,
"website_configuration": null,
"website_url": "http://exmaple-bucket-123123123123.s3-website-us-east-1.amazonaws.com"
}
Finallly the PlanResourceChange_Response_PlannedState resemble everything again:
{
"accelerate_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"access_control": {
"type": 0,
"data": {
"0": 0
}
},
"analytics_configurations": {
"type": 0,
"data": {
"0": 0
}
},
"arn": "arn:aws:s3:::exmaple-bucket-123123123123",
"bucket_encryption": {
"server_side_encryption_configuration": [
{
"bucket_key_enabled": false,
"server_side_encryption_by_default": {
"kms_master_key_id": null,
"sse_algorithm": "AES256"
}
}
]
},
"bucket_name": "exmaple-bucket-123123123123",
"cors_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"domain_name": "exmaple-bucket-123123123123.s3.amazonaws.com",
"dual_stack_domain_name": "exmaple-bucket-123123123123.s3.dualstack.us-east-1.amazonaws.com",
"id": "exmaple-bucket-123123123123",
"intelligent_tiering_configurations": {
"type": 0,
"data": {
"0": 0
}
},
"inventory_configurations": {
"type": 0,
"data": {
"0": 0
}
},
"lifecycle_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"logging_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"metrics_configurations": {
"type": 0,
"data": {
"0": 0
}
},
"notification_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"object_lock_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"object_lock_enabled": {
"type": 0,
"data": {
"0": 0
}
},
"ownership_controls": {
"rules": [
{
"object_ownership": "BucketOwnerEnforced"
}
]
},
"public_access_block_configuration": {
"block_public_acls": true,
"block_public_policy": true,
"ignore_public_acls": true,
"restrict_public_buckets": true
},
"regional_domain_name": "exmaple-bucket-123123123123.s3.us-east-1.amazonaws.com",
"replication_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"tags": {
"type": 0,
"data": {
"0": 0
}
},
"versioning_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"website_configuration": {
"type": 0,
"data": {
"0": 0
}
},
"website_url": "http://exmaple-bucket-123123123123.s3-website-us-east-1.amazonaws.com"
}
According to this doc:
Terraform Core therefore constructs the proposed new state by taking the attribute value from Configuration if it is non-null, and then using the Prior State as a fallback otherwise, thereby helping a provider to preserve its previously-chosen value for the attribute where appropriate.
As such, I expect that PlanResourceChange_Request_ProposedNewState will not set bucket_encryption
and ownership_controls
to null.
Digging through the framework open issues, I found similar problem reported here: https://github.com/hashicorp/terraform-plugin-framework/issues/898
local test confirmed this behavior, after removing the Required : true
and set the Computed: true
+ Optional: true
on the suspected attributes
index 99015cb97..55824e56a 100644
--- a/internal/aws/s3/bucket_resource_gen.go
+++ b/internal/aws/s3/bucket_resource_gen.go
@@ -446,7 +446,9 @@ func bucketResource(ctx context.Context) (resource.Resource, error) {
// Property: SSEAlgorithm
"sse_algorithm": schema.StringAttribute{ /*START ATTRIBUTE*/
Description: "Server-side encryption algorithm to use for the default encryption.",
- Required: true,
+ // Required: true,
+ Optional: true,
+ Computed: true,
Validators: []validator.String{ /*START VALIDATORS*/
stringvalidator.OneOf(
"aws:kms",
@@ -466,7 +468,9 @@ func bucketResource(ctx context.Context) (resource.Resource, error) {
}, /*END SCHEMA*/
}, /*END NESTED OBJECT*/
Description: "Specifies the default server-side-encryption configuration.",
- Required: true,
+ // Required: true,
+ Optional: true,
+ Computed: true,
Validators: []validator.List{ /*START VALIDATORS*/
listvalidator.UniqueValues(),
}, /*END VALIDATORS*/
@@ -2684,7 +2688,9 @@ func bucketResource(ctx context.Context) (resource.Resource, error) {
}, /*END SCHEMA*/
}, /*END NESTED OBJECT*/
Description: "Specifies the container element for Object Ownership rules.",
- Required: true,
+ // Required: true,
+ Optional: true,
+ Computed: true,
Validators: []validator.List{ /*START VALIDATORS*/
listvalidator.UniqueValues(),
Results:
terraform plan
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI configuration:
│ - hashicorp/awscc in /usr/local/go/bin
│
│ The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases.
╵
awscc_s3_bucket.example: Refreshing state... [id=exmaple-bucket-123123123123]
No changes. Your infrastructure matches the configuration.
Community Note
Terraform CLI and Terraform AWS Cloud Control Provider Version
Affected Resource(s)
Terraform Configuration Files
Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.
Debug Output
Panic Output
Expected Behavior
Resource should not be re-deployed every time, seems like there are some changes done on AWS and the provider configuration is not keeping up. Ref. encryption setting from the plan above
Actual Behavior
Re-deployed the same configuration, but the bucket is then deleted and deployed again.
Steps to Reproduce
Apply two times the example in the following documentation: https://registry.terraform.io/providers/hashicorp/awscc/latest/docs/resources/s3_bucket
terraform apply
Important Factoids
References
0000