search

GoogleCloudPlatform / gsutil

A command line tool for interacting with cloud storage services.
Apache License 2.0
875 stars 334 forks source link

gsutil cp with no-clobber option from GCS to S3 returning access denied error #849

Open basava-hnb opened 5 years ago

basava-hnb commented 5 years ago

Hi,

I am using below command to copy files from gcs to s3.

gsutil -m cp -r -c -n <gs://some_bucket/some_files*> <s3://some_bucket/prefix>

This works fine as long as files not present in the destination(s3) i.e., files will be copied if not present. However, if I re-run the same command to copy missing files if any, I am being bombarded with below error for files which are already present but the missing files get copied successfully.

AccessDeniedException: 403 AccessDenied

Per my understanding, -n option should ignore files which are already present in the destination from being overwritten again.

If I do the reverse operation i.e., copying from s3 to gcs, gsutil cp -n command is behaving as expected.

The problem is only when copying from gcs to s3.

Please advise on this

rgurskis commented 4 years ago

Hi @basava-hnb , Looks like it requires s3:GetObjectAcl permissions to use -n option. I was having exactly same problems before this permission was set. Cheers, Rimvis

basava-hnb commented 4 years ago

Thank You Rimvis.

On Wed, Mar 18, 2020 at 6:26 PM rgurskis notifications@github.com wrote:

Hi @basava-hnb https://github.com/basava-hnb , Looks like it requires s3:GetObjectAcl permissions to use -n option. I was having exactly same problems before this permission was set. Cheers, Rimvis

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/gsutil/issues/849#issuecomment-600606973, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALLMERMT2UWX3Z3GWEB5TQLRIDAJLANCNFSM4ICFA5YQ .