Failed to transfer data from s3 bucket with public access to gcs bucket

[Michaelvll]

I have a s3 bucket that has all the ACL permission open to public and make all objects in it public by manually selecting them in the web console (as shown below).

It seems that I can transfer the data from that s3 bucket to a gs bucket by:

gsutil -m cp -r s3://mybucket gs://mybucket-gcp

However, skyplane-nightly complains about the permission issue, as shown in the test.log file (I have hidden the bucket name in the log). I am guessing the proble was caused by the bucket does not have the GetBucketLocation permission, as I did observe that the data transfer service failed as well (unfortunately, I lost the log).

[Michaelvll]

After applying the following policy to the bucket, it seems working now:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "acm-pca.amazonaws.com"
            },
            "Action": [
                "s3:GetBucketAcl",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::mybucket/*",
                "arn:aws:s3:::mybucket"
            ]
        }
    ]
}

[parasj]

@Michaelvll Apologies for the delay on this but it was a pretty major refactor to add support for public buckets. It's now merged to main so the transfer completes successfully. Let me know if there are any issues with it.