Implement vfolder-host access control

[achimnol]

There are some user scenarios that admins want to allow a specific project X members to mount read-only vfolders from the vfolder host A while they can freely create/use vfolders in the vfolder host B—and they want to prohibit the project X member's vfolder creation on the vfolder host A at the same time.

Currently, we only have the per-vfolder access control table (vfolder_permissions) and the access permission to vfolder hosts as a simple boolean set (merged from domains, groups, and keypair_resource_policies). As such, it is impossible to define a "read-only" permission to a specific vfolder host like that users may mount vfolders in the host but may not create new vfolders there.

The problem

Since we have allowed_vfolder_hosts defined in 3 different objects (domains, groups, and keypair_resource_policies), we need to first define how to merge them when each object has different properties like below:

domains("domainA").vfolder_host_permissions = {"storge1": {"access": "rw"}}
groups("groupX").vfolder_host_permissions = {"storage1": {"access": "ro"}}
keypair_resource_policies("researchers").vfolder_host_permissions = {"storage1": {"access": "rw"}}
# or
domains("domainA").vfolder_host_permissions = {"storage1": {"access": "ro"}}
groups("groupX").vfolder_host_permissions = {"storage1": {"access": "rw"}}
keypair_resource_policies("researchers").vfolder_host_permissions = {"storage1": {"access": "rw"}}
# or, when the user belongs to both groupX and groupY
groups("groupX").vfolder_host_permissions = {"storage1": {"access": "ro"}}
groups("groupY").vfolder_host_permissions = {"storage1": {"access": "rw"}}
keypair_resource_policies("researchers").vfolder_host_permissions = {"storage1": {"access": "ro"}}

In the above cases, should what access permission be applied?

Even if we extend the problem scope to a generic RBAC, the same issue arises with multiple subjects sharing the same target object and allowed action lists.

Design (2022/09/30)

ACL checks will be done in the server-side, while the configuration APIs should be exposed to the control panel and webui.

• [ ] Prepare for the generic RBAC/ACL rule checker and the database table
  • Add a new model module: ai.backend.manager.models.acl
  • The acl table schema
  • id: uuid
  • subject_type: str
  • subject_id: uuid (no explicit foreign key yet)
  • target_type: str
  • target_id: uuid (no explicit foreign key yet)
  • allowed_actions: list[str]
  • blocked_actions: list[str]
  • For now, assume that there is only one target_type: "vfolder_host"
• [ ] Implement ACL check functions in the ai.backend.manager.manager.models.acl module
  • check_acl(root_ctx, subject_type, subject_id, target_type, target_id, action) -> bool
  • A simple direct query against the acl table
  • Use begin_readonly() for better concurrency in the database
  • check_merged_acl(root_ctx, subjects: list[tuple[subject_type, subject_id]], target_type, target_id, action) -> bool
  • Calculation logic for when there are multiple subjects (e.g., domain, group, user) applied to the current context:
    • merged_allowed_actions = {take the union of allowed_actions of the all matched rows}
    • merged_blocked_actions = {take the union of blocked_actions of the all matched rows}
    • result = action in (allowed_actions - blocked_actions)
  • Use begin_readonly() for better concurrency in the database
  • Later, after we migrate to ORM-based polymorphic association, we could fold the type-id argument pairs into mapped objects.
  • get_available_actions(target_type) -> list[str]
  • Return the predefined set of available actions for the given target type.
  • Currently, we have only one target type: "vfolder_host" with the following actions:
    • create_vfolder
    • delete_vfolder
    • mount_vfolder
    • list_vfolder
  • As the action names may be added in the future, let's treat them as string constants instead of strict enums. (It's hard to migrate PostgreSQL enums in alembic!)
• [ ] Add a new manager REST API GET /acl/actions?target_type={} to query the available action list based on get_available_actions()
• [ ] Add new manager GraphQL APIs to query and mutate the acl table.
• [ ] Migrate domains.allowed_vfolder_hosts, groups.allowed_vfolder_hosts, keypair_resource_policies.allowed_vfolder_hosts to the acl table using the default: all actions are allowed for each record. Drop the old columns during the alembic migration.
• [ ] Insert calls to check_acl() and check_merged_acl() to the required places including session creation routines (mount_vfolder), and replace all invocations to get_allowed_vfolder_hosts_by_*() functions in ai.backend.manager.models.vfolder with the new check_merged_acl() function, etc.
• [ ] Add the new primary key column as UUID to the domains table to allow reference from acl.subject_id.
• [ ] Migrate the vfolder hosts configurations from etcd to a new vfolder_hosts table defined in ai.backend.manager.models.storage
  • Use UUIDs for the primary key so that the acl.target_id can refer them.
• [ ] Update any required webui/control-panel API calls to query and mutate the vfolder hosts via the generic config APIs to use the new GraphQL APIs implemented above.

Follow-ups

• After merging #698, any modification to the acl table should be logged in the audit logs as well.

[achimnol]

One idea by @fregataa:

• Let's add a property to specify whether "overriding by more specific objects" is allowed or not. (e.g., groups("groupY").vfolder_host_permissions = {"storage1": {"access": "rw", "allow_override": True}})

[achimnol]

Another way is to split the permission level ("ro" or "rw") into fine-grained set of 'atomic' actions like generic RBAC and merge them by taking the union sets:

domains("domainA").vfolder_host_permissions = {"storge1": ["mount"]}
groups("groupX").vfolder_host_permissions = {"storage1": ["mount", "create-vfolder"]}
keypair_resource_policies("researchers").vfolder_host_permissions = {"storage1": ["mount", "create-vfolder", "delete-vfolder"]}
-> merged result: {"storage1": ["mount", "create-vfolder", "delete-vfolder"]}

domains("domainA").vfolder_host_permissions = {"storge1": ["mount", "create-vfolder"]}
groups("groupX").vfolder_host_permissions = {"storage1": ["mount"]}
keypair_resource_policies("researchers").vfolder_host_permissions = {}
-> merged result: {"storage1": ["mount", "create-vfolder"]}

In this case, it is no longer needed to store these information as columns of existing object entities but as a separate RBAC table for improved caching.

[achimnol]

Here is a good discussion about RBAC and ReBAC, with the "role explosion" issue, presented in PyCon KR (Korean): https://youtu.be/DXf_rQtpLZQ

Things to think about:

In this PR's design, we don't have a separate "role" property of an ACL entry but the subject itself may have a collective persona/identity (e.g., a domain applies to all of its users) and the caller of rule checker should provide the full list of multiple, contextual personas.

• How is this design different from the conventional RBAC's static role or dynamic role?
• What's the scalability difference in terms of materialization and calculation?