Fix Authentication issues introduced in 0.14.3, and other minor fixes

Chaffelson commented 3 years ago

nipyapi-0.16.1-py2.py3-none-any.whl.zip

Revert 0.14.3 changes to Authentication handling which introduced basicAuth support but resulted in some NiFi connections appearing incorrectly as Anonymous Added simpler basicAuth control to force it via a config switch without changing tokenAuth and other Authorization header behavior during normal usage nipyapi.config.global_force_basic_auth is now available for use for this purpose Moved all Security controls in config.py to a common area at the foot of the file Removed auth_type from security.service_login as it is now redundant Added controls to handle certificate checking behavior which has become more strict in recently versions of Python3, ssl_verify and check_hostname are now handled security.set_service_auth_token now has an explicit flag for ssl host checking as well Fix oversight where improved model serialisation logic was not correctly applied to Registry Removed unusused parameter refresh from parameters.update_parameter_context Reduced unecessary complexity in utils.dump with no change in functionality Updated client gen mustache templates to reflect refactored security and api client code Minor linting and docstring and codestyle improvements

coveralls commented 3 years ago

Coverage increased (+0.4%) to 69.11% when pulling 546436348e22aa6076a4c71a2eca175b466acf4b on fixAuthc into b0452516d5a902d9d6d1edd02ccd069216bf790c on main.

iMajna commented 3 years ago

Small update regarding PR.

While authentication towards both Nifi and NiFi Registry works now:

Registry Authentication nipyapi.security.get_service_access_status(service='registry') OUTPUT:

{'anonymous': False,
'identity': 'user-1,
'login_supported': True,
'resource_permissions': {'any_top_level_resource': {'can_delete': True,
                                                 'can_read': True,
                                                 'can_write': True},
                      'buckets': {'can_delete': True,
                                  'can_read': True,
                                  'can_write': True},
                      'policies': {'can_delete': True,
                                   'can_read': True,
                                   'can_write': True},
                      'proxy': {'can_delete': True,
                                'can_read': True,
                                'can_write': True},
                      'tenants': {'can_delete': True,
                                  'can_read': True,
                                  'can_write': True}}}

NiFi Authentication nipyapi.security.get_service_access_status(service='nifi') OUTPUT:

{'access_status': {'identity': 'user-1',
               'message': 'You are already logged in.',
               'status': 'ACTIVE'}}

Listing Buckets I can even list buckets: nipyapi.versioning.list_registry_buckets() OUTPUT:

4:{'allow_bundle_redeploy': False,
'allow_public_read': False,
'created_timestamp': 1607535940892,
'description': None,
'identifier': '62878614-fc26-48da-9ca1-9c9d3c3e1255',
'link': {'href': 'buckets/62878614-fc26-48da-9ca1-9c9d3c3e1255',
      'params': {'rel': 'self'}},
'name': 'test1',
'permissions': {'can_delete': True, 'can_read': True, 'can_write': True},
'revision': None}

Importing Flow version

The issue happens when I try to make import of flow from one registry to another:

 nipyapi.versioning.import_flow_version(
        bucket_id=uniqueBucket.identifier,
        encoded_flow=devExport,
        flow_id=prodFlow.identifier
    )

OUTPUT (DEBUG log):

send: b'GET /nifi-registry-api/buckets/62878614-fc26-48da-9ca1-9c9d3c3e1255/flows HTTP/1.1\r\nHost: registry.prod.host:18433\r\nAccept-Encoding: identity\r\nAccept: application/json\r\nContent-Type: application/json\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nAuthorization: Bearer REDACTED_TOKEN2\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 21 Jan 2021 13:37:34 GMT
header: Content-Type: application/json
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate
header: Pragma: no-cache
header: Expires: 0
header: Strict-Transport-Security: max-age=31540000 ; includeSubDomains
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: frame-ancestors 'self'
header: Content-Length: 458
header: Server: Jetty(9.4.19.v20190610)
send: b'GET /nifi-api/system-diagnostics HTTP/1.1\r\nHost: nifi.prod.node:8443\r\nAccept-Encoding: identity\r\nAccept: application/json\r\nContent-Type: application/json\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nAuthorization: Bearer REDACTED_TOKEN\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 21 Jan 2021 13:37:34 GMT
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: frame-ancestors 'self'
header: X-XSS-Protection: 1; mode=block
header: X-Content-Type-Options: nosniff
header: Strict-Transport-Security: max-age=31540000
header: Cache-Control: private, no-cache, no-store, no-transform
header: Server: Jetty(9.4.19.v20190610)
header: X-ProxiedEntitiesAccepted: true
header: Vary: Accept-Encoding
header: Vary: Accept-Encoding
header: Content-Type: application/json
header: Content-Length: 1901
send: b'GET /nifi-registry-api/swagger/swagger.json HTTP/1.1\r\nHost: registry.prod.host:18433\r\nAccept-Encoding: identity\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nContent-Type: application/json\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Thu, 21 Jan 2021 13:37:34 GMT
header: Content-Type: text/plain;charset=iso-8859-1
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate
header: Pragma: no-cache
header: Expires: 0
header: Strict-Transport-Security: max-age=31540000 ; includeSubDomains
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: frame-ancestors 'self'
header: Transfer-Encoding: chunked
header: Server: Jetty(9.4.19.v20190610)
send: b'GET /nifi-registry-api/buckets/62878614-fc26-48da-9ca1-9c9d3c3e1255/flows HTTP/1.1\r\nHost: registry.prod.host:18433\r\nAccept-Encoding: identity\r\nAccept: application/json\r\nContent-Type: application/json\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nAuthorization: Bearer REDACTED_TOKEN2\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 21 Jan 2021 13:37:35 GMT
header: Content-Type: application/json
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate
header: Pragma: no-cache
header: Expires: 0
header: Strict-Transport-Security: max-age=31540000 ; includeSubDomains
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: frame-ancestors 'self'
header: Content-Length: 458
header: Server: Jetty(9.4.19.v20190610)
send: b'GET /nifi-api/system-diagnostics HTTP/1.1\r\nHost: nifi.prod.node:8443\r\nAccept-Encoding: identity\r\nAccept: application/json\r\nContent-Type: application/json\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nAuthorization: Bearer REDACTED_TOKEN\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 21 Jan 2021 13:37:35 GMT
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: frame-ancestors 'self'
header: X-XSS-Protection: 1; mode=block
header: X-Content-Type-Options: nosniff
header: Strict-Transport-Security: max-age=31540000
header: Cache-Control: private, no-cache, no-store, no-transform
header: Server: Jetty(9.4.19.v20190610)
header: X-ProxiedEntitiesAccepted: true
header: Vary: Accept-Encoding
header: Vary: Accept-Encoding
header: Content-Type: application/json
header: Content-Length: 1901
send: b'GET /nifi-registry-api/swagger/swagger.json HTTP/1.1\r\nHost: registry.prod.host:18433\r\nAccept-Encoding: identity\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nContent-Type: application/json\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Thu, 21 Jan 2021 13:37:35 GMT
header: Content-Type: text/plain;charset=iso-8859-1
header: X-Content-Type-Options: nosniff
header: X-XSS-Protection: 1; mode=block
header: Cache-Control: no-cache, no-store, max-age=0, must-revalidate
header: Pragma: no-cache
header: Expires: 0
header: Strict-Transport-Security: max-age=31540000 ; includeSubDomains
header: X-Frame-Options: SAMEORIGIN
header: Content-Security-Policy: frame-ancestors 'self'
header: Transfer-Encoding: chunked
header: Server: Jetty(9.4.19.v20190610)
Traceback (most recent call last):
  File "<string>", line 4, in <module>
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/versioning.py", line 685, in import_flow_version
    nipyapi.utils.validate_parameters_versioning_support()
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/utils.py", line 521, in validate_parameters_versioning_support
    '0.6', service='registry', bool_response=True)
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/utils.py", line 540, in enforce_min_ver
    if check_version(min_version, service=service) == 1:
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/utils.py", line 492, in check_version
    '/swagger/swagger.json', 'GET', _preload_content=False
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/registry/api_client.py", line 332, in call_api
    _return_http_data_only, collection_formats, _preload_content, _request_timeout)
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/registry/api_client.py", line 153, in __call_api
    _request_timeout=_request_timeout)
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/registry/api_client.py", line 355, in request
    headers=headers)
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/registry/rest.py", line 233, in GET
    query_params=query_params)
  File "/opt/virtualenv/python3.6/lib64/python3.6/site-packages/nipyapi/registry/rest.py", line 224, in request
    raise ApiException(http_resp=r)
nipyapi.registry.rest.ApiException: (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 21 Jan 2021 13:37:35 GMT', 'Content-Type': 'text/plain;charset=iso-8859-1', 'X-Content-Type-Options': 'nosniff', 'X-XSS-Protection': '1; mode=block', 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Pragma': 'no-cache', 'Expires': '0', 'Strict-Transport-Security': 'max-age=31540000 ; includeSubDomains', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'Transfer-Encoding': 'chunked', 'Server': 'Jetty(9.4.19.v20190610)'})
HTTP response body: b"Access is denied due to: Unknown user with identity 'anonymous'. Contact the system administrator.\n"

NOTE! All commands are executed in the same session one after another in debug console

Chaffelson commented 3 years ago

Looks like I may have impacted the registry swagger lookup by the auth changes as well, will need to test it.

send: b'GET /nifi-registry-api/swagger/swagger.json HTTP/1.1\r\nHost: registry.prod.host:18433\r\nAccept-Encoding: identity\r\nUser-Agent: Swagger-Codegen/1.0.0/python\r\nContent-Type: application/json\r\n\r\n'
reply: 'HTTP/1.1 401 Unauthorized\r\n'

Chaffelson commented 3 years ago

nipyapi-0.16.1-py2.py3-none-any.whl.zip Updated distributable with fixed Registry Swagger version check

Chaffelson / nipyapi

Fix Authentication issues introduced in 0.14.3, and other minor fixes #256