S3 Bucket Policy JSON always show diff between `pulumi up` and `pulumi refresh`

[guitarrapc]

Language: dotnet (C#)

Description

Even S3 Bucket Policy was applied with pulumi up, pulumi refresh always detect change. Detected change is quite simple, remove eol \n and spaces. Next time I ran pulumi up change will not detect,

NOTE: S3 Bucket Policy is defined as Policy describe, with \n and spaces. pulumi refresh is defer from actual policy.

Expected behavior

no diff reported on pulumi refresh.

Actual behavior

• pulumi refresh report diff. It removes JSON'S \n and space from actual S3 Bucket Policy.

Step to reproduce

Let's create bucket and attach simple policy.

    class Test
    {
        public async Task CreateAsync()
        {
            var bucket = new Bucket("test");
            var policy = new BucketPolicy("test", new BucketPolicyArgs
            {
                Bucket = bucket.Id,
                Policy = bucket.Arn.Apply(async arn => (await GetS3BucketPolicySingle(arn)).Json)
            });
        }
        private Task<GetPolicyDocumentResult> GetS3BucketPolicySingle(string arn)
        {
            var policy = GetPolicyDocument(new[]
            {
                new GetPolicyDocumentStatementsArgs
                {
                    Sid = "Allow bucket write",
                    Effect = "Allow",
                    Actions = new List<string>
                    {
                        "s3:PutObject",
                    },
                    Principals = new List<GetPolicyDocumentStatementsPrincipalsArgs>
                    {
                        new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "Service",
                            Identifiers = new List<string>
                            {
                                "cloudtrail.amazonaws.com",
                            }
                        },
                    },
                    Resources = new List<string> { $"{arn}/*" },
                },
            });
            return policy;
        }

        private static Task<GetPolicyDocumentResult> GetPolicyDocument(GetPolicyDocumentStatementsArgs[] statements, string? version = null, string? policyId = null)
        {
            return Pulumi.Aws.Iam.Invokes.GetPolicyDocument(new GetPolicyDocumentArgs
            {
                Version = version,
                PolicyId = policyId,
                Statements = statements.ToList(),
            });
        }
    }

run pulumi up to apply change.

$ pulumi up

Resources:
    + 2 created

run pulumi refresh will detect change.

$ pulumi refresh

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:master::aws-master::pulumi:pulumi:Stack::aws-master-master]
    ~ aws:s3/bucketPolicy:BucketPolicy: (update)
        [id=test-b2e4753]
        [urn=urn:pulumi:master::aws-master::aws:s3/bucketPolicy:BucketPolicy::test]
        [provider=urn:pulumi:master::aws-master::pulumi:providers:aws::default_1_18_0::f7924a8b-331b-4c9e-98e9-36e2d90d022e]
        --outputs:--
      ~ policy: "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"Allow bucket write\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"s3:PutObject\",\n      \"Resource\": \"arn:aws:s3:::test-b2e4753/*\",\n      \"Principal\": {\n        \"Service\": \"cloudtrail.amazonaws.com\"\n      }\n    }\n  ]\n}" => "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"Allow bucket write\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"cloudtrail.amazonaws.com\"},\"Action\":\"s3:PutObject\",\"Resource\":\"arn:aws:s3:::test-b2e4753/*\"}]}"

apply refresh and run pulumi up or pulumi refresh again will not detect change.

$ pulumi up
Resources:
    1 unchanged
$ pulumi refresh
Resources:
    1 unchanged

Let's change to policy and pulumi up, this detect change on pulumi refresh again.

        private Task<GetPolicyDocumentResult> GetS3BucketPolicySingle(string arn)
        {
            var policy = IamPolicy.GetPolicyDocument(new[]
            {
                new GetPolicyDocumentStatementsArgs
                {
                    Sid = "Allow bucket write",
                    Effect = "Allow",
                    Actions = new List<string>
                    {
                        "s3:PutObject",
                    },
                    Principals = new List<GetPolicyDocumentStatementsPrincipalsArgs>
                    {
                        new GetPolicyDocumentStatementsPrincipalsArgs
                        {
                            Type = "Service",
                            Identifiers = new List<string>
                            {
                                // add this!
                                "logs.ap-northeast-1.amazonaws.com",
                                "cloudtrail.amazonaws.com",
                            }
                        },
                    },
                    Resources = new List<string> { $"{arn}/*" },
                },
            });
            return policy;
        }

$ pulumi up
Resources:
    ~ 1 updated

$ pulumi refresh

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:master::aws-master::pulumi:pulumi:Stack::aws-master-master]
    ~ aws:s3/bucketPolicy:BucketPolicy: (update)
        [id=test-b2e4753]
        [urn=urn:pulumi:master::aws-master::aws:s3/bucketPolicy:BucketPolicy::test]
        [provider=urn:pulumi:master::aws-master::pulumi:providers:aws::default_1_18_0::f7924a8b-331b-4c9e-98e9-36e2d90d022e]
        --outputs:--
      ~ policy: "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"Allow bucket write\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"s3:PutObject\",\n      \"Resource\": \"arn:aws:s3:::test-b2e4753/*\",\n      \"Principal\": {\n        \"Service\": [\n          \"logs.ap-northeast-1.amazonaws.com\",\n          \"cloudtrail.amazonaws.com\"\n        ]\n      }\n    }\n  ]\n}" => "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"Allow bucket write\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":[\"cloudtrail.amazonaws.com\",\"logs.ap-northeast-1.amazonaws.com\"]},\"Action\":\"s3:PutObject\",\"Resource\":\"arn:aws:s3:::test-b2e4753/*\"}]}"

Resources:
    ~ 1 to update

version

    <PackageReference Include="Pulumi.Aws" Version="1.18.0-preview" />
    <PackageReference Include="Pulumi" Version="1.8.1-preview" />

[guitarrapc]

It would be better dotnet pulumi set JSON without \n and white spaces, as pulumi refresh returns, or pulumi refresh should respect what Bucket Policy describe. (Bucket Policy contains \n and spaces. It's not a big impact but non-necessary change detected on refresh confuse me my code is bad or any.

[guitarrapc]

won't happen with latest pulumi 2.3.0 & pulumi-aws 2.8.0. close issue.