Proposal for default permissions with scratch bucket

[choldgraf]

Background and proposal

Background We use scratch buckets as temporary places to store cloud data to facilitate various data operations with intermediate representations. These buckets are generally cleared after a few days, since they're not meant for long-term storage.

Currently, users have the ability to read and write to the scratch bucket. However, they don't have the permission to delete from the scratch bucket.

Proposed solution Our default should be to give users read, write, and delete access. We should document that there are some risks associated with this, and if a community representative wants to restrict delete access, they can request it to happen.

Implementation guide and constraints

Here's some sample language for a policy / guidelines to use for our current practices, including the suggestion that we give delete privileges to users by default.

By default, your community's scratch bucket will have read/write/delete permissions for any users on the hub. **Do not store sensitive data in scratch** if you don't want others to be able to view or edit it. If you wish to prevent users from manually deleting files across the scratch bucket, it is possible to enable this, but may result in errors for users that (intentionally or not) attempt to over-write files in scratch.

Updates and ongoing work

No response

[sgibson91]

I just want to flag that while this has the effect of "just" adding the delete permission, what it looks like in practice in the GCP console is giving the service account the admin role. There is no specific delete role, you just have to grant them the admin role.

Ref: https://cloud.google.com/storage/docs/access-control/iam-roles

• Before: Pangeo's service account had Storage Object Viewer and Creator
• After: it has Storage Object Admin (I also removed Creator and Viewer as they are just subsets of Admin)

In light of this, I think we should do one of two things:

1. Go ahead with the proposed policy after satisfying ourselves that Admin doesn't have more permissions than the combination of Creator+Viewer than we'd like; or
2. Reverse the proposed policy: By default, users will have restricted read/write permissions to the scratch bucket, but privileged permissions (including delete) will not be unreasonably withheld upon request with a valid use case.

[damianavila]

My internal ❤️ is screaming for option 2 😜 given that admin privileges seem, maybe, too much power? 🤔

[choldgraf]

If admins privileges come with extra permissions, then I agree with @damianavila . Can we document a list of those extra permissions? We'll need to share this with community reps to help them make this decision anyway

[sgibson91]

I would love to create a little Markdown table of what the differences between Creator+Viewer and just Admin are, but I feel I have a lot on my plate right now in terms of big picture reasoning about things. Very happy for someone else who has some extra brain cycles to take on that work.

I think it would involve expanding the wildcard at the end of storage.objects and storage.multipartUploads and seeing if there's anything destructive that is included in Admin but not Creator+Viewer.

[sgibson91]

While I also agree that it would be good to have something more easier to parse than those docs to present to community reps, we also need to be careful that it doesn't get scale with Google's docs too quickly.