S3 PutBucketPolicyCommand returns 'Malformed JSON' error

[brmur]

Describe the bug A clear and concise description of what the bug is. Attempting to add an S3 bucket policy results in V3 (using code below) results in "Error MalformedPolicy: Policies must be valid JSON and the first byte must be '{' " Identical JSON in V3 succeeds.

SDK version number

    "@aws-sdk/client-s3": "^1.0.0-gamma.1",

Is the issue in the browser/Node.js/ReactNative? Node.js

Details of the browser/Node.js/ReactNative version v12.14.1

To Reproduce (observed behavior) Steps to reproduce the behavior (please share code or minimal repo) V3 version (returns malformed error)

// Import an S3 client
const {
  S3Client,
  CreateBucketCommand,
  PutBucketWebsiteCommand,
  PutBucketPolicyCommand,
} = require("@aws-sdk/client-s3");

// Set the AWS Region
const REGION = "eu-west-1"; //e.g. "us-east-1"

// Create params JSON for S3.createBucket
const bucketName = "BUCKET_NAME"; //BUCKET_NAME
const bucketParams = {
  Bucket: bucketName,
};

var readOnlyAnonUserPolicy = {
  Version: "2012-10-17",
  Statement: [
    {
      Sid: "AddPerm",
      Effect: "Allow",
      Principal: "*",
      Action: ["s3:GetObject"],
      Resource: [""],
    },
  ],
};

// create selected bucket resource string for bucket policy
var bucketResource = "arn:aws:s3:::" + "BUCKET_NAME" + "/*"; //BUCKET_NAME
readOnlyAnonUserPolicy.Statement[0].Resource[0] = bucketResource;

// convert policy JSON into string and assign into params
var bucketPolicyParams = {
  Bucket: "BUCKET_NAME",
  Policy: JSON.stringify(readOnlyAnonUserPolicy),
};

// Instantiate an S3 client
const s3 = new S3Client(REGION);

const run = async () => {
  try {
    const response = await s3.send(
      new PutBucketPolicyCommand(bucketPolicyParams)
    );
    console.log("Success, permissions added to bucket", response);
  } catch (err) {
    console.log("Error", err);
  }
};
run();

V2 version (works)

var AWS = require("aws-sdk");
// Set the region
AWS.config.update({ region: "eu-west-1" });

// Create S3 service object
s3 = new AWS.S3({ apiVersion: "2006-03-01" });

var readOnlyAnonUserPolicy = {
  Version: "2012-10-17",
  Statement: [
    {
      Sid: "AddPerm",
      Effect: "Allow",
      Principal: "*",
      Action: ["s3:GetObject"],
      Resource: [""],
    },
  ],
};

// create selected bucket resource string for bucket policy
var bucketResource = "arn:aws:s3:::" + "BUCKET_NAME" + "/*";
readOnlyAnonUserPolicy.Statement[0].Resource[0] = bucketResource;

// convert policy JSON into string and assign into params
var bucketPolicyParams = {
  Bucket: "BUCKET_NAME",
  Policy: JSON.stringify(readOnlyAnonUserPolicy),
};

// set the new policy on the selected bucket
s3.putBucketPolicy(bucketPolicyParams, function (err, data) {
  if (err) {
    // display error message
    console.log("Error", err);
  } else {
    console.log("Success", data);
  }
});

Expected behavior For both to work.

[AllanZhengYP]

@brmur

Can you print the the request body by:

bucketPolicyParams.middlewareStack.add(next => args => {
  console.log("request body: ", args.request.body);
  return next(args);
})

It's expected to be a valid xml string.

[brmur]

@AllanFly120 returns 'Error TypeError: Cannot read property 'add' of undefined'

[lucasmerlin]

I tried using puBucketPolicy with minio and I'm experiencing the same issue.

This is the request body being sent: <?xml version="1.0" encoding="UTF-8"?><PutBucketPolicyRequest xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Policy>{"Version":"2012-10-17","Statement":[{"Sid":"AddPerm","Effect":"Allow","Principal":"*","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::test/*"]}]}</Policy></PutBucketPolicyRequest>

My guess is, that instead of a xml string this should be only the json object?

The aws s3 api also says it should be a json body: https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketPolicy.html

That would also explain the error message @brmur is getting.

[AllanZhengYP]

Thank you all raising this to us. I have confirmed that this results from the discrepency of the Smithy model(even in the newest version) and the JSON model that was used in V2. The Policy member should be marked as payload instead of document: https://github.com/aws/aws-sdk-js-v3/blob/f95cce338fcdc49ead6e3ca6d178a6fd58ae556f/codegen/sdk-codegen/aws-models/s3.2006-03-01.json#L8450-L8455 (by adding "smithy.api#httpPayload": {}).

We will reach out to service team to have the model updated.

[AllanZhengYP]

Hi @lucasmerlin @brmur

The the service model has been updated. The change will land in the SDK in the next service model release. I will let you know when the SDK containing the change is released.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.