Cannot update parameter context on NiFi 1.15.0

gareth625 commented 2 years ago

Nipyapi version: 0.17.1
NiFi version: 1.15.0
NiFi-Registry version: N/A
Python version: 3.9.5 (also tested with 3.8.10)
Operating System: MacOS Monterey and Docker version 20.10.8, build 3967b7d

Description

Describe what you were trying to get done, or what you would like the package to do.
Tell us what happened, what went wrong, and what you expected to happen.

I am trying to update an existing parameter context using the nipyapi.parameters.update_parameter_context function. The code I am using works with NiFi 1.14.0 but does not work with NiFi 1.15.0. Running again 1.15.0 I get an HTTP 500 error. I saw no logs in the NiFi 1.15.0 container.

What I Did

import urllib3
from nipyapi import config, parameters, security, utils
from pprint import pprint
from uuid import uuid4

def create_and_update_parameters(port: int, username: str, password: str):
    config.global_ssl_verify = False
    config.nifi_config.verify_ssl = False
    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
    utils.set_endpoint(f"https://localhost:{port}/nifi-api")
    security.service_login(service='nifi', username=username, password=password)

    context_name = 'test_context'
    context_description = 'Context to test the updates with'
    context = parameters.get_parameter_context(context_name, greedy=False)
    if not context:
        context = parameters.create_parameter_context(
            context_name, context_description, parameters=[])
    print('Initial parameter context')
    pprint(context)

    param_0_value = str(uuid4())
    param_0 = parameters.prepare_parameter(
        'test_param_0', param_0_value, 'First test parameter', sensitive=False
    )
    params = [param_0]
    context.component.parameters = params
    updated_context = parameters.update_parameter_context(context)
    print('Context update with new parameters:')
    pprint(params)
    pprint(updated_context)

print('-'*16)
print('Updating parameter context on NiFi 1.14.0')
port_1140 = 8414
username_1140 = 'fe877546-e75f-46a4-bb99-6a251f350130'
password_1140 = 'GJJFQKjlAtBs189NHzXEv3FNrEarF6GC'
create_and_update_parameters(port_1140, username_1140, password_1140)

print('-'*16)
print('Updating parameter context on NiFi 1.15.0')
port_1150 = 8415
username_1150 = 'e28d7579-1250-4864-ba2a-1c975d5c5618'
password_1150 = 'wRa5lbfWGVqmfIj8q3BGDoTR+rQwAdCh'
create_and_update_parameters(port_1150, username_1150, password_1150)

nipyapi.nifi.rest.ApiException: (500) Reason: Internal Server Error HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 25 Nov 2021 20:05:17 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-XSS-Protection': '1; mode=block', 'X-Content-Type-Options': 'nosniff', 'Strict-Transport-Security': 'max-age=31540000', 'Set-Cookie': '__Secure-Request-Token=eac4ec50-2791-4e57-963b-6c6213c341a2; Path=/; Secure', 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT', 'Content-Type': 'text/plain', 'Vary': 'Accept-Encoding', 'Content-Encoding': 'gzip', 'Content-Length': '88', 'Server': 'Jetty(9.4.44.v20210927)'}) HTTP response body: An unexpected error has occurred. Please check the logs for additional details.

I saw no logs in docker-compose logs -f nifi-1150 when I ran the command. I didn't try and increase any log levels.

I used the following docker-compose YAML to start my two NiFi instance and got the autogenerated passwords from the logs. Next time I'll play with your Docker classes ;)

services:
  nifi-1140:
    image: apache/nifi:1.14.0
    environment:
      - NIFI_JVM_HEAP_INIT=2g
      - NIFI_JVM_HEAP_MAX=8g
      - NIFI_WEB_HTTPS_PORT=8414
    ports:
      - "8414:8414"
  nifi-1150:
    image: apache/nifi:1.15.0
    environment:
      - NIFI_JVM_HEAP_INIT=2g
      - NIFI_JVM_HEAP_MAX=8g
      - NIFI_WEB_HTTPS_PORT=8415
    ports:
      - "8415:8415"

Urgency

Not very, we are prototyping nipyapi code to automate interactions with our NiFi instance, just updated to 1.15.0, but have nothing deployed yet and won't until the new year. It won't block our use of nipyapi but will limit how we use it.

Chaffelson commented 2 years ago

Thanks for providing the detailed issue @gareth625 , I'm testing the 1.15.0 swagger client now, there's some minor bugs but it's looking good so far

gareth625 commented 2 years ago

I've run the above script using git+https://github.com/Chaffelson/nipyapi.git@nifi-1-15 as the pip requirement and I can update the NiFi 1.15.0 instance parameter context. It does look good so far :)

Chaffelson commented 2 years ago

Should be fixed now in release 0.18.0

gareth625 commented 2 years ago

It is fixed as far as I can tell. Should I close this? I'm happy for you to.

Chaffelson / nipyapi