TypeError using get_access_policy_for_resource

[Amorik]

• Nipyapi version: 0.9.1
• NiFi version: 1.6.0/1.7.0
• NiFi-Registry version: N/A
• Python version: 3.6.3
• Operating System: CentOS 7.4.1708

Description

In our current docker build/setup we are experiencing some issues with proper policies rights being set. We sought to use nipyapi to correct that. While learning/understanding how to create/modify/delete policy information in NiFi we attempted to use the nipyapi function "nipyapi.security.get_access_policy_for_resource" to view existing information. However using the call, regardless of the variations in options, fails to complete, errors on "TypeError: Got an unexpected keyword argument 'id' to method get_access_policy_for_resource".

What I Did

The test script:

nipyapi.nifi.configuration.verify_ssl = False
nipyapi.config.nifi_config.host = 'https://nifi:9443/nifi-api'
nipyapi.security.set_service_ssl_context(
    service='nifi',
    ca_file='nifi-cert.pem',
    client_cert_file='client-cert.pem',
    client_key_file='client-key.key',
    client_key_password='supersecretpassword'
)
canvas_root_policy = nipyapi.security.get_access_policy_for_resource('flow', 'read')
print(canvas_root_policy)

Fails with the following output:

Traceback (most recent call last):
  File "./test.py", line 37, in <module>
    canvas_root_policy = nipyapi.security.get_access_policy_for_resource('flow', 'read')
  File "/opt/cgimss-utils/lib/python3.6/site-packages/nipyapi/security.py", line 367, in get_access_policy_for_resource
    id=r_id
  File "/opt/cgimss-utils/lib/python3.6/site-packages/nipyapi/nifi/apis/policies_api.py", line 279, in get_access_policy_for_resource
    (data) = self.get_access_policy_for_resource_with_http_info(action, resource, **kwargs)
  File "/opt/cgimss-utils/lib/python3.6/site-packages/nipyapi/nifi/apis/policies_api.py", line 314, in get_access_policy_for_resource_with_http_info
    " to method get_access_policy_for_resource" % key
TypeError: Got an unexpected keyword argument 'id' to method get_access_policy_for_resource

When lookin into the code a bit we can see the call being made to "nipyapi.nifi.PoliciesApi().get_access_policy_for_resource" contains the keyword "id". Following it down to where the error comes from it appears it is just because that key is not listed in "all_params" on line 303 of "policies_api.py". Adding it correct's the behavior, though may not be the right way to fix the issue. It might also appear in other similar calls?

nipyapi/security.py

    360     try:
    361         if service == 'nifi':
    362             log.info("Getting NiFi policy for %s:%s/%s",
    363                      action, resource, r_id)
    364             return nipyapi.nifi.PoliciesApi().get_access_policy_for_resource(
    365                 action=action,
    366                 resource=resource,
    367                 id=r_id
    368             )

nipyapi/nifi/apis/policies_api.py

    303         all_params = ['action', 'resource']
    304         all_params.append('callback')
    305         all_params.append('_return_http_data_only')
    306         all_params.append('_preload_content')
    307         all_params.append('_request_timeout')
    308
    309         params = locals()
    310         for key, val in iteritems(params['kwargs']):
    311             if key not in all_params:
    312                 raise TypeError(
    313                     "Got an unexpected keyword argument '%s'"
    314                     " to method get_access_policy_for_resource" % key
    315                 )
    316             params[key] = val
    317         del params['kwargs']

Urgency

Currently in development, nothing is in production today. Started to use nipyapi to automate some initial deployment tasks. We encountering issues with Docker NiFi and policies, hoping to fix them with nipyapi. There are manual work arounds for the time being.

[Chaffelson]

Hi @Amorik Anything under nipyapi/nifi or nipyapi/registry is procedurally generated from the swagger definition in the main NiFi projects. What this generally means is, while we can change them locally, there's an underlying reason for the mismatch. This then tends to mean one of three things:

• I've screwed up and broken the function definition somehow
• The swagger definition doesn't quite match the actual interface (happens rarely)
• They've changed something upstream and the client is out of sync (more likely)

I haven't updated the client for the 1.7.0 release yet (working on it), but I will add investigating this to my list for it. If you are interested in investigating this yourself then the process for generating the swagger definition from the main NiFi project, and then base python client, are in the NiPyApi dev notes

[Chaffelson]

@Amorik I've updated NiPy to support NiFi-1.7.1, can you please test nipyapi-0.10.0 and let me know if the ID issue is resolved here?

[Amorik]

@Chaffelson issue is not resolved. Same error. Tried 0.10.0 and 0.10.1

[Chaffelson]

Thanks for testing - I'll take another look with @kevdoran

[Chaffelson]

Right, I've reproduced your issue and I think I've found the cause and a possible solution. It looks like in <=NiFi-1.5.0 that call does in fact require an id parameter, but in 1.6.0> it has been deprecated (and now works the same as the NiFi-Registry call).

I think I can fix this by implementing something I've been considering that I'd need for a while, which is version-specific NiFi calls. I'll need to think about the best way to implement this properly, but I'll mock up a fix now to test whether my theory is correct.

[Chaffelson]

Hi @Amorik @kevdoran and I chased this down - I believe that I introduced the problem in 0.9 and hadn't noticed, but it should now be fixed in 0.10.2 if you would care to try it.

[Amorik]

@Chaffelson confirmed, working! Sorry it took me a while to get to it. I haven't done anything advanced with it yet but the script above works using 0.10.2. Now i can look at automating Authorizations, woot!

[Chaffelson]

Thanks for taking the time to test it - I'll close this issue now but reopen it if you need to.