authzed / spicedb

Open Source, Google Zanzibar-inspired database for scalably storing and querying fine-grained authorization data
https://authzed.com/docs
Apache License 2.0
5.16k stars 281 forks source link

Proposal: Support ABAC dynamically without hard code criteria in relation caveat, or provide caveat context in authn call #1966

Open RevenChen-083 opened 4 months ago

RevenChen-083 commented 4 months ago

Problem Statement

I'm implementing ABAC auth check with caveat feature, like "only employees with [env=prod,region=sg] can access servers in Singapore production environment". In Netflix sample case, resource attributes to be checked are specified in relation caveat, while user attributes are provided via caveat context in CheckPermission endpoint. But I have some concerns when defining auth check caveats.

Suppose we have a hierarchical server structure and organization structure scenario like this:

resource group -------- assigned access to ----->> user group
    │                                                  │
    └── server                                         └── user group
    └── server                                                 └── user

Where:

Then:

  1. IMHO, I don't think it's a good idea to maintain attributes to be checked in caveat definition or relation tuple like this, cuz if resource attribute is changed, then schema or assigned relation tuple needs to be changed accordingly. That's pretty weird after changing attribute, resource group owner need to notify authn admin to revoke & re-grant authorization. Authn admin would like to reduce his workload by making authn policies, not authn detailed items

    image
  2. Authz check related attributes vary from case to case, I don't want to define several caveats for several cases.

  3. I think it's inappropriate to run auth check based on input attribute values in caveat context, cuz authz caller doesn't know (or even doesn't care) what attribute the server to be checked is carrying. Besides, provided context might not be trustworthy, authz system should work as a standalone system which only depends on authn truth stored internally or verified data.

Solution Brainstorm

I have an idea about problems above:

  1. Introduce entity tag concept to spicedb, each resource_type:resource_id can have tag KV pairs associated
  2. Introduce tag inheritance mechanism, to introduce tag from relation subject/object. Like "server could inherit env and region tag from "belongs_to" relation related resource groups"
  3. Introduce tag fetching functions and value comparison functions in caveat execution(cel), based on specified tag names in relation tuple, fetch tag values in caveat execution process for further comparison. For example, suppose authn policy is "user in user_group:ug1 can access servers which have same env&region label in resource_group:rg1", we can define caveat and relations like this:
caveat dynamic_abac(tags_to_be_checked list<string>) {
  is_any_tag_matched(tags_to_be_checked)
}

// tag inheritance rule
server inherits tag [env,region] from relation [belongs_to] as object
user inherits tag [env,region] from relation [member] as subject

// tag
resource_group:rg1 has region:sg
server:srv1 has env:prod
user_group:dev has env:prod
user:user1 has region:sg

// relation tuples
server:srv1#belongs_to@resource_group:rg1
user_group:dev#member@user:user1
resource_group:rg1#access@user_group:dev#member[dynamic_abac:{"tags_to_be_checked": ["env","region"]}]

pseudocode for is_any_tag_matched:

func is_any_tag_matched(tags []string) bool {
  for tag in tags {
    objValues := getObjectTag(tag)
    subjValues := getSubjectTag(tag)

    if ! objValues isAnyIn subjValues {
      return false
    } 
  }
  return true
}

This function is provided is provide by a cel library loaded at run time, which carries subject and object info in function closure. getObjectTag, getSubjectTag and isAnyIn functions are exposed for caveat usage as well.

Then authn caller would just need to call check permission server:srv1 access user:user1, without fetching resource tags and passing them via caveat context param.

Besides, we can tell other fancy stories based on this, for example:

  1. Authn admin can assign auth to different people in different ways without changing schema, like this:
    
    // user in dev group can only access servers in the same env and region
    resource_group:rg1#access@user_group:dev#member[dynamic_abac:{"tags_to_be_checked": ["env","region"]}]

// sg sre user can access servers in the sg region, but not in jp region resource_group:rg1#access@user_group:sre#member[dynamic_abac:{"tags_to_be_checked": ["region"]}]

// root user can access all servers in any env or region resource_group:rg1#access@user_group:root#member[dynamic_abac:{"tags_to_be_checked": []}]


2. In [Authzed playground](https://play.authzed.com/relationships), example schema "Google IAM in SpiceDB", a `role_binding` is assigned to `spanner_database`, bound with `user` and `role`. With features mentioned above, authn admin can assign auth statically specifically, by maintain tags on `role_binding`, or assign auth generally like usually, without change any schema definition

caveat dynamic_abac(tags_to_be_checked list) { is_any_tag_matched(tags_to_be_checked) }

// tag inheritance rule user_group inherits tag [ip] from role_binding as subject

// tag role_binding:specific_binding has ip:127.0.0.1 server:srv1 has ip:127.0.0.1

// dev user can only access server with IP 127.0.0.1 as authn admin specified role_binding:specific_binding#user@user_group:dev resource_group:rg1#granted@role_binding:specific_binding[dynamic_abac:{"tags_to_be_checked": ["allowed_ip"]}]

// sre user can access servers in the same region role_binding:specific_binding#user@user_group:sre resource_group:rg1#granted@role_binding:specific_binding[dynamic_abac:{"tags_to_be_checked": ["region"]}]



Now the development is done, and I'm running tests against it locally, I would like to share this idea with you and contribute if it's useful to the community as well. If you have any suggestions/questions/concerns, please just let me know.
vroldanbet commented 4 months ago

Hi, before really diving into solution ideation, I think we need to clarify the problem to solve.

IMHO, I don't think it's a good idea to maintain attributes to be checked in caveat definition or relation tuple like this, cuz if resource attribute is changed, then schema or assigned relation tuple needs to be changed accordingly. That's pretty weird after changing attribute, resource group owner need to notify authn admin to revoke & re-grant authorization. Authn admin would like to reduce his workload by making authn policies, not authn detailed items

It's a tradeoff. One of the important aspects of caveat design is that it is safely typed. When an auditor reads the schema, they can understand what the behaviour is. As you can see in the blog post, we discussed a completely dynamic caveat versus the current implementation, and we though the latter is safer. We understand not all use-cases can be implemented when you don't know ahead of time the set of attributes. This is why we implemented isSubtreeOf, because you can create a hierarchical, untyped tree using map. Netflix had the same issue - they didn't know the tags ahead of time, and we initially discussed implementing dynamic caveats for them, but landed on the usage of comparing trees of key/values.

So I don't necessarily agree this is "not a good idea" nor "weird". SpiceDB was designed from the ground up to be "safe" and hard to make mistakes in the schema language, and the current caveat design follows that philosophy.

Authz check related attributes vary from case to case, I don't want to define several caveats for several cases.

I think that's understandable if you have an unbounded set of restrictions that you cannot predict upfront, hence the alternative idea of dynamic caveats. The idea is to basically provide a CEL expression as caveat argument, but you still need to define arguments based on the type system. We use this approach in some of our commercial features, but the functionality is not available upstream.

I think it's inappropriate to run auth check based on input attribute values in caveat context, cuz authz caller doesn't know (or even doesn't care) what attribute the server to be checked is carrying. Besides, provided context might not be trustworthy

I'm confused by the usage of "auth" and "authn" in the context of authorization operations. are you talking about authentication here?

I also don't agree with this take. A service has to talk to SpiceDB, you need to provide arguments, so there is already a level of trust established, regardless there are caveats involved or not. You cannot invoke authorization operations without providing arguments, whether that's caveat context or the rest of the RPC payload. And it's important to call out that nothing prevents the client application from completely ignoring what SpiceDB says and make their own authorization decisions. So you need to consider what your threat model is, and if you don't trust your own application, your probably need to look into a different strategy.

authz system should work as a standalone system which only depends on authn truth stored internally or verified data.

This is categorically stating that things should be one way. I don't think the reality is like that. I'd suggest defining a threat model and using that to inform what the solution space would look like.

The reality is that not all data can be stored because it's dynamic. Think of all the metadata that comes as part of an HTTP request, you could make authorization decisions based on the HTTP verb, the IP, time of the day, geolocation, etc. Caveats was designed to handle dynamic data like that.

RevenChen-083 commented 4 months ago

Hi Victor, thanks for your reply.

First of all, I might need to clarify some terms:

  1. By authn, I mean auth admin grant authentication to someone. And by authz I mean verify user authorization at run time.

  2. By "dynamic", I don't mean "have expressions defined at run time" like the blog mentioned or "provide a CEL expression as caveat argument" as you mentioned above, caveat statements/params should be defined in schema explicitly for type safety/auditing purpose, I totally agree with that. What I mean is instead of defining attribute check statement in caveat, defining expected values in relation, providing actual attribute values at run time, I prefer to just define attribute names to be checked in caveat, at run time, with newly integrated attribute value fetching functions, attribute values are fetched from internal storage. Like the example shown in solution part:

    
    caveat dynamic_abac(tags_to_be_checked list<string>) {
    is_any_tag_matched(tags_to_be_checked)
    }

// relations // user in dev group can only access servers in the same env and region resource_group:rg1#access@user_group:dev#member[dynamic_abac:{"tags_to_be_checked": ["env","region"]}]

// sg sre user can access servers in the sg region, but not in jp region resource_group:rg1#access@user_group:sre#member[dynamic_abac:{"tags_to_be_checked": ["region"]}]

// pseudocode for is_any_tag_matched func is_any_tag_matched(tags []string) bool { for tag in tags { objValues := getObjectTag(tag) subjValues := getSubjectTag(tag)

if ! objValues isAnyIn subjValues {
  return false
} 

} return true }


And I might have different opinions about:
> A service has to talk to SpiceDB, you need to provide arguments, so there is already a level of trust established, regardless there are caveats involved or not. You cannot invoke authorization operations without providing arguments, whether that's caveat context or the rest of the RPC payload. And it's important to call out that nothing prevents the client application from completely ignoring what SpiceDB says and make their own authorization decisions. So you need to consider what your threat model is, and if you don't trust your own application, your probably need to look into a different  strategy.

Yes arguments should be provided to SpiceDB when you talks to it, but here's a question, can callers provide these arguments?
1. In the scenario I mentioned, authz callers know nothing about attributes on resources/users, or authentication caveat details, they doesn't know what should be provided or from where to get these arguments.
2. From my previous working experience, we have a system design that security system only trust verified encrypted ticket, which is issued by gateway security plugins or other security components, containing data like IP, verified identity etc. In authorization check calls, we decrypt that ticket, derive info we want, run auth check and issue another ticket/token afterwards, with that, applications can call other services/APIs protected by security system. Thus applications are forced to call SpiceDB-like authz system, without providing arguments they know nothing about.

Sure we can add a wrapper or adapter to SpiceDB which is responsible to provide attrs in caveat context during authz call, then define caveats like Netflix's case, then we go back to
> Authz check related attributes vary from case to case, I don't want to define several caveats for several cases.  

For example, for SRE colleagues, they can access all services in the same region, while developers can only access servers which in environment developers are assigned. With current SpiceDB features, I must define two caveats, one takes expected and actual env&region as parameter, another one takes expected and actual region as parameter. Or one complex caveat like:

caveat match_fine( expected_env list, expected_regions list, observed_account string, observed_region string ) { observed_region in expected_regions && (len(expected_env) ==0 || observed_env in expected_env) }

server:srv#access@user:sre[match_fine:{"expected_env":[],"expected_regions":["sg"]}] server:srv#access@user:dev[match_fine:{"expected_env":["prod"],"expected_regions":["sg"]}]


Neither defining two caveats nor one complex looks like an elegant solution to clients, not to mention relation must be adjusted when attribute on server changes.

> The reality is that not all data can be stored because it's dynamic. Think of all the metadata that comes as part of an HTTP request, you could make authorization decisions based on the HTTP verb, the IP, time of the day, geolocation, etc. Caveats was designed to handle dynamic data like that.

I totally agree with you about this. Context info should be provided in caveat context, while about attr-like persistent data, anyway they should be stored somewhere. Given that these attributes are quite close to authorization, and resources might inherit/override attributes from related resources, so I think store attributes in SpiceDB is reasonable, to enhance SpiceDB's capabilities in ABAC.

SpiceDB is a great implementation of Google Zanzibar model, what I'm trying to do is provide embedded attribute fetching capabilities, to encapsulate authentication details as much as possible, keep caveat definitions and relations as stable as possible, as simple as possible when community developers want to implement ABAC model based on it.
vroldanbet commented 4 months ago

@RevenChen-083,

I don't have enough context about your business domain to be able to provide more feedback. But what the things that I think I gather from your comment are:

If the information is not dynamic at all and is not provided at request time because it can be stored, then caveats is not the right tool for the job here: the core SpiceDB's relationship-based paradigm should be used to model your business domain, which means introduce all those tags as first-class definition. Yes, that includes concepts like "env", "region", "account".

The problem with caveats is that it's very easy to pattern-match with previous experiences (e.g. if you are using a "descendant of XACML" - anything that looks like a traditional policy engine), and folks may miss the powerful relationship-based paradigm at the core of SpiceDB. I'm not saying you requirements can be definitely implemented here, but I'd probably start there and see if there is anything in the core language (i.e. everything but caveats) that can be augmented to support your use-case before jumping to caveats. I certainly recall working with other community members in relatively similar domains that didn't have to resource to caveats.

RevenChen-083 commented 4 months ago

Hi Victor, I think we are on the same page about my case now :) Actually I explored similar ways to implement my case based on existing SpiceDB capabilities before I came up with my own idea, unfortunately I cannot persuade myself to take any of them...

  1. Do it as Netflix sample blog. As I mentioned several times above, attributes and relations are maintained by different teams, storing attrs in relation tuples might cause data inconsistency between my clients and me, we'll both suffer from keeping data in sync.

  2. Define attr as first class citizen definition, then maintain relation like env:prod#belongs_to@server:srv or server:srv#contains@env:prod, that could be an option of how attributes be stored, but it doesn't help with auth check... I don't know how to define a meaningful directed relation-based path from resource to user, with attributes relation involved.

One possible solution would be running auth check with SpiceDB, then run additional attribute check in wrapper system. Sounds possible, while in my case, resources could "inherit" attributes from its related resources, relationships are stored in SpiceDB, I have to do sth similar as SpiceDB does, to iterate relation paths and get values I want. I just couldn't help to have a feeling that sth is not done in SpiceDB properly, so I have to finish its job somewhere else.

it is a bit of a departure from the overall design of the system because nodes in the graph don't exist in isolation - they exist when defined by a relationship. It's also a departure in terms of Zanzibar decomposing the evaluation of the graph as subproblems, which are now composed of relationships and k/v that could be retrieved when evaluating caveats at any point in the graph. Something like that requires careful study to make it fit into the current philosophy of the project.

I do respect the philosophy of the project and agree with the importance of subproblem decomposing, I just don't see why introducing attr k/v would harm that. Now I can create a relation tuple in SpiceDB even if the entity don't exist at all, still I can create attr k/v for non-exist entity. I can delete one entity from its source system, then notify SpiceDB to delete relevant relations (or just do nothing, leave them there), so as attr k/v! Attr kv's can be handled in the same way as relations did. I just don't see any conflicts/differences between these two concepts. Providing suck solutions (sorry to use it but just as you state) to ABAC scenarios to keep relation as the only one first class citizen doesn't sound like a wise choice.

And by the way we are even discussing about node entity lifecycle management internally to avoid updating all affected relations when an entity node is deleted. By saving "entity deleted at zedtoken/transactions" info internally, then in relation searching process we can check relation's created transaction against subject&object's delete transaction, relations with lower transaction are filtered cuz they are not valid anymore.

It's also a departure in terms of Zanzibar decomposing the evaluation of the graph as subproblems, which are now composed of relationships and k/v that could be retrieved when evaluating caveats at any point in the graph.

I'm not suggesting "evaluate caveats at any point in the graph" either, it is still evaluated at the very end of check function call. Get attrs for origin input subject&objects then run value comparison accordingly. There's no need to care about attr kv's or caveats when solving subproblem, just like before.

The problem with caveats is that it's very easy to pattern-match with previous experiences (e.g. if you are using a "descendant of XACML" - anything that looks like a traditional policy engine), and folks may miss the powerful relationship-based paradigm at the core of SpiceDB.

ReBAC auth model has its own beauty, I don't think people who choose Spicedb would ignore that paradigm and just use it to build traditional auth engine. But still, some tricks could be taken from traditional engines to make SpiceDB always easy to use, not only when you use it carefully and laboriously right? At least duplicate attrs in in relation tuple is not good enough.

If possible, I would like to join your further discussion with other community members about this topic (or similar topics) Have a good one :)

vroldanbet commented 4 months ago

Define attr as first class citizen definition, then maintain relation like env:prod#belongs_to@server:srv or server:srv#contains@env:prod, that could be an option of how attributes be stored, but it doesn't help with auth check... I don't know how to define a meaningful directed relation-based path from resource to user, with attributes relation involved.

Can you clarify what you mean with auth check here? I'm not sure if you mean authentication or authorization.

I don't understand enough of your business domain to be able to help. Based on some of the information you wrote in the discussion body, I created this playground example:

I do respect the philosophy of the project and agree with the importance of subproblem decomposing, I just don't see why introducing attr k/v would harm that.

I am not saying it compromises that, but it requires carefully thinking about how to fit it to retain the system's scaling properties.

ReBAC auth model has its own beauty, I don't think people who choose Spicedb would ignore that paradigm and just use it to build traditional auth engine.

I talk from my experience as a maintainer. I don't know anything about the person on the other end so I can only try to take a step back and see what problem they are trying to solve, just like I'm doing here.

RevenChen-083 commented 4 months ago

Hi Victor,

Thank you for showing me that example, relation-based attr maintenance does work perfectly for that simple case, while in my case:

  1. Entities might have multilpe attr values, like biz_line=[ai_training,ai_application], and attr value comparison logic varies from client to client when multiple attr values with the same key assigned. Some might say user's attrs must be superset of attr value, while some might say one of user's attr values is in server's attr value set is good enough. I'm afraid it cannot be implemented with relation-based attr maintenance.

  2. Attributes are NOT mandatory to entities. (If presents, then check, otherwise just let it pass)

  3. To be checked attr categories vary as well... As I mentioned above, for dev users, they can access server only when env and region attr both match with server, while for sre users, only region attr is required to be checked.

Besides I'm afraid schema definition like that is not intuitive enough, at least it might take me a while to come up with a schema like attribute relates to users, and user can access servers only when he's in user set intersection of server's region & env attr related userset.

about how to fit it to retain the system's scaling properties.

Maybe it should not be a problem I guess? (Please correct me if I'm wrong or something I don't see) Attributes are fetched in caveat execution for just one time, at the very end of auth check, so attr data just need to follow the same data-partitioning policy as relations. If all SpiceDB nodes hold complete data copy, good, nothing to worry about; If relations are saved in partitions(like by entity type and ID), so does attributes! Attribute fetching could also be fanned out / re-distributed to where attrs are stored.

I fully understand as a project maintainer, you need to think carefully before introducing something new, definitely as a developer using SpiceDB I should thank you for that. :) About the requirement we're discussing about, I guess both of us could have some take-aways. Anyway I do hope we can have native solutions to such requirement in SpiceDB in the future lol, let's keep in touch if any idea pops up.

vroldanbet commented 4 months ago

Besides I'm afraid schema definition like that is not intuitive enough, at least it might take me a while to come up with a schema like attribute relates to users, and user can access servers only when he's in user set intersection of server's region & env attr related userset

I understand that a relationship-based model takes a while to get used to, but I believe there is a payoff to the investment. I'm probably not the most representative user of the system, since I maintain it, I get that, but after some training, it comes naturally.

Your use case seems to heavily lean into policy and ABAC and it's not fitting well with the currently supported set of features, although there are things we could do to address that, I'm wondering what drove you to start with a relationship-based access control model in the first place. What specific aspects of the model do you think were a good fit for the problems you are facing? Perhaps you were looking into the ability scale across the globe like Zanzibar, or have tunable consistency parameters because your application demands it? Perhaps the ability to scale horizontally?

Maybe it should not be a problem I guess? (Please correct me if I'm wrong or something I don't see) Attributes are fetched in caveat execution for just one time, at the very end of auth check, so attr data just needs to follow the same data-partitioning policy as relations. If all SpiceDB nodes hold complete data copy, good, nothing to worry about; If relations are saved in partitions(like by entity type and ID), so are attributes! Attribute fetching could also be fanned out / re-distributed to where attrs are stored.

Stored caveat context is part of the relationship, so as the graph is traversed, relationships and their context are loaded. It flows naturally with the execution engine and the way problems are decomposed. To load the data, we probably want to eagerly load the additional data as part of the relationship load, and we need to do it on a single round-trip, so we'd need to JOIN the relationships resource and subject type/ID pars with another table to enrich the context with data coming from the "entity context storage". SpiceDB does not do any JOINs at the moment, and that could be problematic with distributed databases like CockroachDB and Spanner (SpiceDB has to support multiple databases) where JOIN may require cross-node coordination and would almost certainly add latency.

The alternative to piggybacking into the current relationship queries is to do follow-up queries, and you can either do that on every relationship query (probably a bad idea) or at the time the collected composed CEL expression is evaluated in the entry-point node with the API call input context. This would add just one additional roundtrip, but one of Zanzibar's core design principles is to avoid as much database access as possible doing all sorts of tricks, but mostly reusing work that has been already done. At that point we've lost information on the relationships that have been evaluated - all we have is a "smushed/flattened" composed CEL expression, and we've lost information on the relationships whose "centralized context" you should have loaded in the first place. Turning that into internal API dispatch metadata will also be problematic, adding more overhead to dispatching API.

I hope this sheds some light on the complex machinery inside. There is a reason caveats were designed to be stored along the relationships, as it fits well with the core design principles. I think adding a centralized caveat context store for entities is an interesting idea to explore and could make some use cases simpler to manage.

I appreciate the discussion and the insight into the problems you are trying to solve. I'll open a feature request to gather more feedback from the community.

EDIT: Opened https://github.com/authzed/spicedb/issues/1981

RevenChen-083 commented 4 months ago

Hi Victor,

Thanks for the clarifications above. At the first place, my team chose ReBAC and SpiceDB because we were building a centralized authorization platform which serves multiple independent biz applications, ReBAC model fits in many of them, the ability to explain why authorization is granted or how it could be granted can also provide productive features to our users. And as a centralized platform, horizontal scaling must be considered as well. About consistency, that's also a plus in some cases. While as business grows, now we need to handle attribute & attribute inheritance cases in authorization checks.

Stored caveat context is part of the relationship, so as the graph is traversed, relationships and their context are loaded. It flows naturally with the execution engine and the way problems are decomposed. To load the data, we probably want to eagerly load the additional data as part of the relationship load, and we need to do it on a single round-trip, so we'd need to JOIN the relationships resource and subject type/ID pars with another table to enrich the context with data coming from the "entity context storage". SpiceDB does not do any JOINs at the moment, and that could be problematic with distributed databases like CockroachDB and Spanner (SpiceDB has to support multiple databases) where JOIN may require cross-node coordination and would almost certainly add latency.

I fully agree table join is not a good idea in such a latency-sensitive system.

The alternative to piggybacking into the current relationship queries is to do follow-up queries, and you can either do that on every relationship query (probably a bad idea) or at the time the collected composed CEL expression is evaluated in the entry-point node with the API call input context. This would add just one additional roundtrip, but one of Zanzibar's core design principles is to avoid as much database access as possible doing all sorts of tricks, but mostly reusing work that has been already done. At that point we've lost information on the relationships that have been evaluated - all we have is a "smushed/flattened" composed CEL expression, and we've lost information on the relationships whose "centralized context" you should have loaded in the first place. Turning that into internal API dispatch metadata will also be problematic, adding more overhead to dispatching API.

Yes loading entity attribute data would definitely increase API latency, but if we take a look at the whole auth check system call chain, someone, either client or auth system has to get it loaded. The time cost is inevitable from end-to-end point of view, but only if authorization grantor uses ABAC-like features. I understand caveat evaluation with attributes causes one addition relation graph traversal for attr data, but again, only ABAC-like-auth-check-enabled cases are affected. Pure ReBAC and "static caveat" cases are not touched at all.

(My humble opinion just as an application developer: if something has to be done, then just pick the right guy to do it. With attr features encapsulated in auth system, auth check callers/attr maintainers/authorization grantors could be decoupled completely from each other, that's a win-win solution in my case)