search

Chaffelson / nipyapi

A convenient Python wrapper for Apache NiFi
Other
245 stars 76 forks source link

nipyapi.security.get_access_policy_for_resource does not work #110

Closed jrittenh closed 5 years ago

jrittenh commented 5 years ago

Description

Calling nipyapi.security.get_access_policy_for_resource(action=<action>, resource=<resource>) returns None. Looking deeper, the r_id is never appended to the resource. Looking deeper than that, if you append resource + '/' + r_id, nipyapi.nifi.PoliciesApi().get_access_policy_for_resource attempts to escape the slash, and the API returns HTTP response body: The request was rejected because the URL contained a potentially malicious String "%2F".

Traceback (most recent call last):
  File "./initialize_nifi.py", line 296, in <module>
    action=action,
  File "/media/code/nipyapi/nipyapi/nifi/apis/policies_api.py", line 279, in get_access_policy_for_resource
    (data) = self.get_access_policy_for_resource_with_http_info(action, resource, **kwargs)
  File "/media/code/nipyapi/nipyapi/nifi/apis/policies_api.py", line 368, in get_access_policy_for_resource_with_http_info
    collection_formats=collection_formats)
  File "/media/code/nipyapi/nipyapi/nifi/api_client.py", line 326, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/media/code/nipyapi/nipyapi/nifi/api_client.py", line 153, in __call_api
    _request_timeout=_request_timeout)
  File "/media/code/nipyapi/nipyapi/nifi/api_client.py", line 349, in request
    headers=headers)
  File "/media/code/nipyapi/nipyapi/nifi/rest.py", line 233, in GET
    query_params=query_params)
  File "/media/code/nipyapi/nipyapi/nifi/rest.py", line 224, in request
    raise ApiException(http_resp=r)
nipyapi.nifi.rest.ApiException: (500)
Reason: Server Error
HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 07 Feb 2019 21:20:22 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'Content-Length': '87', 'Server': 'Jetty(9.4.11.v20180605)'})
HTTP response body: The request was rejected because the URL contained a potentially malicious String "%2F"

I haven't tested this, but I suspect this may be due to the following code in __call_api:

    # path parameters
    if path_params:
        path_params = self.sanitize_for_serialization(path_params)
        path_params = self.parameters_to_tuples(path_params,
                                                collection_formats)
        for k, v in path_params:
            # specified safe chars, encode everything
            resource_path = resource_path.replace(
                '{%s}' % k, quote(str(v), safe=config.safe_chars_for_path_param))

Urgency

Moderate - continued development is hindered by needing to find a workaround to this issue.

Chaffelson commented 5 years ago

That looks moderately annoying to be sure @jrittenh I'm tied up prepping a workshop for next week today, but I'll try to take a run at it on the weekend. I recall that we wrote those functions for the beta release of registry to be cross compatible, and I haven't revisited them, so this may be a good opportunity for improvement.

ajayjoe commented 5 years ago

I have the same issue trying to add policy for the root canvas. Not able to get the AccessEntity for policy for modify component. So not able to add user permission to modify component.

Chaffelson commented 5 years ago

So I have found a bug in the create_access_policy as well while investigating this: Here's the fixed function, note the 'r_id if r_id else resource' addition

def create_access_policy(resource, action, r_id=None, service='nifi'):
    """
    Creates an access policy for the given resource, action and optionally
    resource_id for NiFi.

    Args:
        resource (str): a valid resource type for this service, e.g. 'bucket'
        action (str): a valid action type for this service, typically 'read',
            'write' or 'delete'
        r_id (optional[str]): if NiFi, the resource ID of the resource
        service (str): the service to target

    Returns:
        An access policy object for that service

    """
    assert isinstance(resource, six.string_types)
    assert action in _valid_actions
    assert r_id is None or isinstance(r_id, six.string_types)
    assert service in _valid_services
    if resource[0] != '/':
        resource = '/' + resource
    try:
        if service == 'nifi':
            return nipyapi.nifi.PoliciesApi().create_access_policy(
                body=nipyapi.nifi.AccessPolicyEntity(
                    revision=nipyapi.nifi.RevisionDTO(version=0),
                    component=nipyapi.nifi.AccessPolicyDTO(
                        action=action,
                        resource=resource + '/' + r_id if r_id else resource
                    )
                )
            )
        # elif service == 'registry':
        return nipyapi.registry.PoliciesApi().create_access_policy(
            body=nipyapi.registry.AccessPolicy(
                action=action,
                resource=resource
            )
        )
    except nipyapi.nifi.rest.ApiException as f:
        log.info("Policy creation unsuccessful, raising error")
        raise ValueError(f.body)
ghost commented 5 years ago

Hi Chaffelson,

we are getting error in get_access_policy_for_resource for component level, global access level works fine. We are passing paramters like below: nipyapi.security.get_access_policy_for_resource('/process-groups/800ccad5-0169-1000-ffff-ffff962cd5ad',action='read')

If the issue is fixed please let us know what was the correction.

Chaffelson commented 5 years ago

I patched a bug in the main function for resources that only take a 'read' policy, I haven't gotten to the root of the rest of the policy controls yet. Fortunately I have the makings of a test setup for it on the go, so we should be able to improve the main security functions significantly shortly.

Chaffelson commented 5 years ago

OK so I've been working this through with a couple of other members of the NiFi community, a couple of positives first:

The bad news is I think we've hit a bug on how the swagger spec is defined for the rest api, in that it's got optional path params on the NiFi side that swagger doesn't like in the default client. I'm loath to hack the client and would rather resolve it elegantly, so we'll try to find a path forward here.

If you're interested, the general error is that the swagger spec defines a 'resource' to be passed, but that resource sometimes contains a literal '/' which gets encoded as %2f instead of being passed through as a literal character. It seems that OpenAPI explicitly doesn't support not encoding a slash, and NiFi rejects encoded slashes as malicious characters. Fun!

Anyway, we're looking into it. Sorry for the bother.

Chaffelson commented 5 years ago

This should now be fixed in the NiFi191 branch

ghost commented 5 years ago

I tried, but could still getting error nipyapi.security.get_access_policy_for_resource(action='read', resource='/process-groups', r_id = '900bfd7e-0169-1000-0000-000052630ddb',service='nifi')

also tried

nipyapi.security.get_access_policy_for_resource(action='read', resource='/process-groups/900bfd7e-0169-1000-0000-000052630ddb', r_id = '900bfd7e-0169-1000-0000-000052630ddb',service='nifi')

can you please help on this. If any implementation has changed. I have checked the documentation for this.

Chaffelson commented 5 years ago

Fixed in release 0.13, please reopen if there's still issues

Chaffelson commented 5 years ago

@saharannaveen1 can you please try with the new 0.13 release and let me know?

ghost commented 5 years ago

@saharannaveen1 can you please try with the new 0.13 release and let me know?

Hi Chaff, it seems to work, but i will let u know once i implement it to add user, usergroup to access polic for component and global level.

Thanks for fixing the bug.

jrittenh commented 4 years ago

It's possible I'm doing something wrong, but my tests seem to be less optimistic. If I run get_access_policy_for_resource while iterating over all combination of process groups and actions, not a single one returns a result. The same call run manually (requests.get(req_url, headers={'Authorization': 'Bearer ' + nipyapi.config.nifi_config.api_key['tokenAuth']}) returns results, which I am able to then build into a barebones AccessPolicyEntity:

action_policy = requests.get(req_url, headers={'Authorization': 'Bearer ' + nipyapi.config.nifi_config.api_key['tokenAuth']})
action_policy_dict = json.loads(action_policy.text)
nipyapi.nifi.AccessPolicyEntity(
    revision=nipyapi.nifi.RevisionDTO(version=action_policy_dict['revision']['version']),
    id=action_policy_dict['id'],
    component=nipyapi.nifi.AccessPolicyDTO(
        id=action_policy_dict['id'],
    ),
)

I've tried v.0.13.0, v0.14.1, and master, all with the same result ("None").

Chaffelson commented 4 years ago

Ok, it sounds like another issue then rather than something I've introduced in the latest update. Unfortunately I'm on isolation parenting at the moment but I'll try to figure this out when time allows.

On Fri, 27 Mar 2020, 20:40 Justin, notifications@github.com wrote:

It's possible I'm doing something wrong, but my tests seem to be less optimistic. If I run get_access_policy_for_resource while iterating over all combination of process groups and actions, not a single one returns a result. The same call run manually (requests.get(req_url, headers={'Authorization': 'Bearer ' + nipyapi.config.nifi_config.api_key['tokenAuth']}) returns results, which I am able to then build into a barebones AccessPolicyEntity:

action_policy = requests.get(req_url, headers={'Authorization': 'Bearer ' + nipyapi.config.nifi_config.api_key['tokenAuth']}) action_policy_dict = json.loads(action_policy.text) nipyapi.nifi.AccessPolicyEntity( revision=nipyapi.nifi.RevisionDTO(version=action_policy_dict['revision']['version']), id=action_policy_dict['id'], component=nipyapi.nifi.AccessPolicyDTO( id=action_policy_dict['id'], ), )

I've tried v.0.13.0, v0.14.1, and master, all with the same result ("None").

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/Chaffelson/nipyapi/issues/110#issuecomment-605305346, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZAZOCCROHLDDSMYDWA5CLRJUFLPANCNFSM4GVIJPYA .