buckyos / CYFS

CYFS is the next-generation technology to build real Web3 by upgrading the basic protocol of Web (TCP/IP+DNS+HTTP),is short for CYberFileSystem. https://www.cyfs.com/, cyfs://cyfs/index_en.html.
https://www.cyfs.com/
BSD 2-Clause "Simplified" License
1.99k stars 276 forks source link

Improve permission system that supports dynamic tokens for DEC #175

Closed lurenpluto closed 1 year ago

lurenpluto commented 1 year ago

The current permission system has two layers: rmeta and object access, with the following characteristics:

These two levels of permissions can handle many static configurations but are insufficient for some dynamic permission scenarios, such as:

A DEC needs to share a specific resource with others and sets a token (maybe a random password string). As long as others know this token, they can access the resource. The shared token has an expiration time, after which others will no longer have access.

This is a typical token-based sharing mechanism, with tokens being custom-generated and validated by each DEC. Tokens can be a random password or a public key format, etc.

The protocol stack needs to consider how to support this form of dynamic permission. It is reasonable to add it to the rmeta layer, as introduced from the beginning. Since DEC validation is required, the handler system also needs to be involved. The DEC can dynamically handle permission validation requests for each rmeta to determine whether to grant or deny access.

lurenpluto commented 1 year ago

On an existing basis codes with ACL and protocols, consider a version of the design as follows:

1. Add handler permission to rmeta

The upgraded access types are as follows:

#[derive(Clone, Debug, Serialize, Deserialize, Eq, PartialEq)]
pub enum GlobalStatePathGroupAccess {
    Specified(GlobalStatePathSpecifiedGroup),
    Default(u32 /*AccessString*/),
    Handler, // New Type, should be handled by the dec's acl handler
}

2. Reuse the previous handler.acl event

This event once played the role of acl callback in the old ACL mechanism, but after the ACL was restructured into a new logic, this event has been deprecated but still retained. We can reactivate this event, but the Request needs to be redesigned and should include the following fields:

pub struct AclHandlerRequest {
    pub dec: ObjectId,
    pub path: String,
    pub source: RequestSourceInfo,
    pub permissions: AccessPermissions,
}

3. rmeta management

We can use add_access and remove_access to set the handler permission for a global-state path. When a request needs to use rmeta for verification, it matches the corresponding path. If it corresponds to a handler, it will call the ACL event registered by the corresponding dec for dynamic judgment, thus determining whether the request is rejected or accepted.

4. Custom token for requests

Since the ACL handler is a general rmeta processing mechanism, all places that need to use rmeta to handle permissions need to support custom tokens. It is somewhat difficult to recklessly add a field to many different request modes, so after researching, we consider adding it to the composite structure of req-path:

https://github.com/buckyos/CYFS/blob/8f8e88c2089d3a5573c2e71504020642b9ffdcfd/src/component/cyfs-lib/src/prelude/global_state_common.rs#L28

RequestGlobalStatePath

Since req-path itself is a composite structure, you can refer to the design pattern of URLs, providing parameters in the form of ? followed by the parameter, like:

{dec-id}/a/b/c?token=xxx

However, note the following two points:

streetycat commented 1 year ago

Great!

Is there some built-in authentication strategy? compare password or verify the signature?

By the way, I think we should use uniform names for some special identifiers.

For example: the dec in AclHandlerRequest, I also find dec_id used in other structures (eg: TypelessObjectDesc.dec_id).

and there are req_path and rpath already existed. what is the path in AclHandlerRequest mean?

lurenpluto commented 1 year ago

Great!

Is there some built-in authentication strategy? compare password or verify the signature?

By the way, I think we should use uniform names for some special identifiers.

For example: the dec in AclHandlerRequest, I also find dec_id used in other structures (eg: TypelessObjectDesc.dec_id).

and there are req_path and rpath already existed. what is the path in AclHandlerRequest mean?

The token should be more precisely called user_token, and it should be transparent to the cyfs-stack. The specific verification mechanism depends on the app's own handling(during acl handler callback). It can use a password form or asymmetric key verification such as a signature. Cyfs-base provides some utility functions and classes for these purposes.

The AclHandlerRequest mentioned above is just pseudocode for illustration purposes, where the path refers to the req-path and dec refers to the dec-id. In the actual implementation, this will be unified.

lurenpluto commented 1 year ago

rmeta adds support for dynamic permissions

1. Add Handler type to the permission type of rmeta

If the corresponding permission type is Handler when adding rmeta access, then when rmeta access check, if it matches the rule, it will try to call the corresponding dec registered acl handler callback, and the return value of the callback method will determine whether the permission request is passed or denied.

rmeta related is defined as follows: https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/component/cyfs-lib/src/rmeta/def/access.rs#L91-L96

Usage as the following code: https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/tests/cyfs-stack-test/src/case/acl_handler.rs#L73-L83

2. Refactoring the acl events in the Router Handlers event system

The new acl event is adapted to the rmeta callback request, the corresponding request and response are defined as follows:

dec needs to check the permission request inside the handler based on the following params:

Here is a complete req_path with query_string

req_path = /a/b/c?token=123456

The request can be answered by the Acl handler with the following AclAction:

https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/component/cyfs-lib/src/acl/action.rs#L12-L16

3. How to register acl events for Dec service

Registering acl events is similar to registering post-object events, you can filter the corresponding rmeta access callback events by filter and req_path

The filter supports the following fields and types:

- Request side

FieldName Type
source.protocol String
source.dec_id String
source.zone String
source.device String
source.zone_category String
dec_id String
req_path Glob
req_query_string Glob
permissions String

- Response side

FieldName Type
action String

The sample code for registering an event is as follows

- The acl handler callback:

https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/tests/cyfs-stack-test/src/case/acl_handler.rs#L20-L58

- Register acl handler:

https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/tests/cyfs-stack-test/src/case/acl_handler.rs#L85-L95

4. How to use the custom req_query_string on the request side

The req_query_string is a custom parameter for the rmeta permissions dynamic Handler, which can be passed with various pre-designed parameters from dec to better verify the permissions of the request.

For all requests that use req_path to rely on rmeta permissions, you can specify additional req_query_string to specify additional parameters for the permission request, such as token, password, etc., including the following scenarios

- NON/NDN/Crypto/Trans etc.

Directly attach a query_string to the req_path of the request: {req_path}? {query_string}

- Global-state based on op-env/accessor's related interface

directly on the path/inner_path parameter of the request, in the following format: {path}? {query_string}

- r-link

You need to include additional query pairs in the query string of the url, but you need to avoid some parameters reserved by the protocol

The current list of reserved parameters is as follows:

For example, here is an example of an r link withtoken=123456 query string:

cyfs://r/$/{dec-id}/test/dynamic/call?token=123456

- o-link

Since the req-path of the o link is itself provided as a query string parameter, the req_path with user-defined query_string needs to be encoded to escape the ? and & special characters to avoid not being recognized correctly by cyfs-stack internally

For example, the following o link:

cyfs://o/{device-id}/{object-id}?req_path=/a/b%3Ftoken=xxx%26id=xxx&dec_id=xxx

where got the decoded parameters are as follows

lurenpluto commented 1 year ago

The dynamic acl base on rmeta handler has been merged into the main, and a simple test case is in

https://github.com/buckyos/CYFS/blob/afd05ebc6788781fedb2ddbea6b7541b0397472b/src/tests/cyfs-stack-test/src/case/acl_handler.rs

The detailed usage documentation is above, hopefully providing some detailed testing help @lizhihongTest

lizhihongTest commented 1 year ago

How can I set cyfs.GlobalStatePathGroupAccess.Handler() by cyfs.AccessString ? If I put_object or publish_file can set access, But the request params type is cyfs.AccessString

lurenpluto commented 1 year ago

The current permission system in cyfs-stack is divided into two layers:

lizhihongTest commented 1 year ago

The interface of dynamic tokens for DEC has been test finished, The remaining r/a-link related tests will be manually performed in the browser.

Testing report

http://bdttest.tinyappcloud.com/cyfs_test_package/test_report/2023_05_04_dynamic_token/

Testing code

cyfs-test-lab:test_dynamic_token_scenario.ts

Testing result

Scenario Testing: Dynamic token Testing register different Acl Handler √ Normal Case: Share object with dynamic token,Another user get_object_by_path with correct token (702 ms) √ Normal Case: Share object with dynamic token,Another user get_object_by_path with incorrect token (769 ms) Testing parameter chain : Just use cyfs.RouterHandlerChain.Acl will take effect √ Normal Case:set chain cyfs.RouterHandlerChain.Acl (598 ms) √ Abnormal Case: set chain cyfs.RouterHandlerChain.NDN will not take effect (551 ms) √ Abnormal Case: set chain cyfs.RouterHandlerChain.Handler will not take effect (559 ms) √ Abnormal Case: set chain cyfs.RouterHandlerChain.PostCrypto will not take effect (575 ms) √ Abnormal Case: set chain cyfs.RouterHandlerChain.PostRouter will not take effect (553 ms) √ Abnormal Case: set chain cyfs.RouterHandlerChain.PreNOC will not take effect (531 ms) Testing parameter default_action : just run effect when routine is error . developer just can use routine to verify token √ Abnormal Case: Not use routine acl handler,default_action set Reject (556 ms) √ Abnormal Case: Not use routine acl handler,default_action set Default Reject (553 ms) √ Abnormal Case: Not use routine acl handler,default_action set Drop (542 ms) √ Abnormal Case: Not use routine acl handler,default_action set Response,but empty response.Acl handler must use routine handler (605 ms) √ Normal Case: Not use routine acl handler,default_action set Pass will use next routine handler will accept (627 ms) √ Abnormal Case: Not use routine acl handler,default_action set Pass will use next routine handler will reject (572 ms) Testing parameter req_path : handler will take effect on the req_path and child req_path √ Normal Case: handler will take effect on the req_path (626 ms) √ Normal Case: handler will take effect on the child req_path (573 ms) √ Abmormal Case: handler will not take effect on the brother req_path (535 ms) √ Abmormal Case: handler will not take effect on the parent req_path (618 ms) Testing parameter index : The smaller the index value, the higher the priority √ Normal Case: set index 0 handler accept,set index 1 handler reject. Result access accept (619 ms) √ Normal Case: set index 1 handler accept,set index 0 handler reject. Result access reject (528 ms) Testing parameter filter[Unknown the filter rules?] : Only the handlers that meet the filter rules will take effect √ Normal Case: set filter match success (608 ms) √ Normal Case: set filter match failed (549 ms) √ Normal Case: set filter = undefined (616 ms) Testing parameter id :The id is the unique identifier of the handler √ Normal Case: Registering a handler with the same id will overwrite it,Set access Accept -> Reject (545 ms) √ Normal Case: Registering a handler with the same id will overwrite it,Set access Reject -> Accept (623 ms) Testing parameter routine : the routine will register a handler event ,When call it,it can response accept、reject、BuckyError √ Normal Case: Registering a handler the routine will response accept (574 ms) √ Normal Case: Registering a handler the routine will response reject (649 ms) √ Normal Case: Registering a handler the routine will response BuckyError (651 ms) √ Normal Case: Registering a handler the routine will return BuckyError (300101 ms)

lizhihongTest commented 1 year ago

Test environment

OOD : Nightly 1.1.0.755 cyfs-runtime : bulid by code c477a173

Random test data code

cyfs-test-lab:test_dynamic_token_r_a_link.ts

Test Data

R-link get object frist time use with correct token

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

R-link get object second time

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

R-link get object second use flags=1 refresh object

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&flags=1

Test Result:

R-link get object second set ketwords params

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&flags=1&page_index=1

Test Result:

R-link get object use error token

Tesk Link: cyfs://r/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=error-hc1Rf0ndAdddcG1S2iic&flags=1

Test Result:

O-link get object frist time

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

O-link get object second time

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic

Test Result:

O-link get object second use flags=1 refresh object

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&&flags=1

Test Result:

O-link get object second set ketwords params

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=sk-hc1Rf0ndAdddcG1S2iic&page_index=1&flags=1

Test Result:

O-link get object use error token

Tesk Link: cyfs://o/5aSixgLox5CRLLaMcnsRz75A4khSXay5WRjfN5VjNNVx/9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?req_path=/9tGpLNndR5tyui8DkYBpEz8mFHzjfqkCVmsFusa5roHd/qa_test_token/share-9tGpLNnLxfgtzB9VkdteBhsdcuFw7FnK69F5twFQ6Z24?token=error-hc1Rf0ndAdddcG1S2iic&&flags=1

Test Result: