cvdfoundation / open-images-dataset

Open Images is a dataset of ~9 million images that have been annotated with image-level labels and bounding boxes spanning thousands of classes.
https://github.com/openimages/dataset
986 stars 157 forks source link

Add 's3:GetBucketLocation' permission to AWS bucket #20

Closed wpjohnson closed 5 years ago

wpjohnson commented 5 years ago

Can you please add the IAM Permission 's3:GetBucketLocation' in an IAM Bucket Policy to the bounding box annotated dataset hosted on AWS S3 (s3://open-images-dataset)?

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicPermissions",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::open-images-dataset"
            ]
        }
    ]
}

Given the current 'public-read' ACL present on the bucket, adding this permission doesn't expose any information that isn't already available. Specifically, the 's3:GetBucketLocation' permission only gives the bucket's 'x-amz-region', which is already available from HEAD Bucket requests and used under-the-hood by the aws cli.

Granting this permission will allow for the use of Google Cloud Storage Transfer Service to copy this dataset from AWS S3 to Google Cloud Storage (see my response in a separate open issue).

wpjohnson commented 5 years ago

@tylin, as the most recent contributor to this repo, can you add this permission?

tylin commented 5 years ago

@wpjohnson, I added the permission. Could you give it a try for storage transfer to GCP?

wpjohnson commented 5 years ago

Just ran a transfer successfully using GCSTS, and it works! Thanks!