Closed saif003 closed 1 year ago
Evidently Boto tries to make a connection to lookup the canned ACL name, and that's where the problem is coming from. An easy workaround would be to just use a file to set the ACL. Is that possible for your workflow? If not, I might suggest using the ch
sub-command, as that allows for setting ACLs without a file or canned ACLs.
thank you @thomasmaclean for the help, I wasn't aware that ch command can be used as an alternative. I replaced above command with following and its working well now.
gsutil -m acl ch -u AllUsers:READ gs://bucket/**
I've an existing setup in which I'm using workload identity federation (WIF) to authenticate circleCI with GCP and everything has been working perfectly fine. Its a simple workflow which uses
gsutil -m rsync -d -r folder/ gs://bucket
command to sync a folder with a GCS bucket.I recently modified my workflow to also run following additional command right after rsync which is basically intended to mark all objects in the bucket as public. I know I can instead mark the whole bucket as public but without going into much details there is a specific reason I'm doing it this way.
gsutil -m acl set -R -a public-read gs://bucket
after making above change, I see following error in circleCI when this command is executed:
the error message is confusing since it says its an authentication issue but I know authentication is not the issue since the rsync command right before this is working fine. I also modified the workflow to do
gCloud auth list
before ACL set and that command also shows that gCloud is authenticated.I know its also not an authorization issue since the service account which is being used by WIF has "Storage Admin" and "Storage Object Admin" roles which adds storage.buckets. and storage.objects. permissions to the account so it has already more than the required permissions needed to set public ACL on the bucket or its objects, you can verify what permission are required for
gsutil acl set
vs what are already there using following documentation links.https://cloud.google.com/storage/docs/access-control/iam-gsutil
https://cloud.google.com/storage/docs/access-control/iam-roles
Surprisingly if I remove WIF authentication and If I directly use a service account key for authentication the error goes away and
gsutil acl set
works fine, which tells me there might be an issue with WIF configuration but nothing seems out of the ordinary to me. I followed this blog post by circleCI to setup OIDC authentication / WIF for GCP. The issue seems to be specific to thegsutil acl set
command, as other gsutil commands (like rsync) are working fine with WIF authentication. I don't want to use service account keys for authentication since google recommends against using them as they can pose security risk if compromised.What I've tried so far,
private
ACL instead ofpublic-read
just to make sure its not an issue specific to specific ACL