treeverse / lakeFS

lakeFS - Data version control for your data lake | Git for data
https://docs.lakefs.io
Apache License 2.0
4.35k stars 346 forks source link

Docs: Describe which permissions are required on GCS env #8091

Open N-o-Z opened 3 weeks ago

N-o-Z commented 3 weeks ago

Recently a user tried to use lakectl with pre-signed urls and encountered an error:

upload folder/example.pdf failed: get physical address to upload object: http2: response body closed

From the lakeFS logs we can see:

time="2024-08-22T02:47:25Z" level=error msg="error generating pre-signed URL" func="pkg/block/gs.(*Adapter).GetPreSignedURL" file="build/pkg/block/gs/adapter.go:239" error="unable to sign bytes: googleapi: Error 403: Permission 'iam.serviceAccounts.signBlob' denied on resource (or it may not exist).\nDetails:\n[\n  {\n    \"@type\": \"type.googleapis.com/google.rpc.ErrorInfo\",\n    \"domain\": \"iam.googleapis.com\",\n    \"metadata\": {\n      \"permission\": \"iam.serviceAccounts.signBlob\"\n    },\n    \"reason\": \"IAM_PERMISSION_DENIED\"\n  }\n]" host=<snipped> method=GET operation_id=GetPhysicalAddress path="<snipped out of this example>&presign=true" user=plus-dev

The user was following the GCP deployment guide We should add a section describing required permissions as we do for AWS

kujenga commented 3 weeks ago

Thanks for filing this.

From https://cloud.google.com/storage/docs/access-control/signing-urls-with-helpers I needed to add the roles/iam.serviceAccountTokenCreator role in addition to bucket permissions.

First example here was also a useful reference: https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/service_account_access_token