Registry import_flow_version calls nifi-api, cannot use library with registry only.

mixam85 commented 2 years ago

Nipyapi version: 0.18.0
NiFi version: 1.14.0
NiFi-Registry version: 0.8.0
Python version: 30.10.2
Operating System: Arch Linux

Description

I would like to be able to use this library with Nifi Registry only. Registry only operations are also sending nifi-api requests.

What I Did

When using nipyapi.versioning.import_flow_version, the code also calls nifi-api : /nifi-api/system-diagnostics. This comes from validate_parameters_versioning_support function which connects to nifi-api & nifi-registry whatever the command called.

Urgency

In our environment nifi and nifi-registry is not deployed on the same host, thus needing for the script to know both server instead of using localhost.

Chaffelson commented 2 years ago

Really good point about it causing a dependency which should really be optional, or at least fail elegantly if nifi is not connected. Let me think about it a bit if you don't already have a workaround to suggest.

mixam85 commented 2 years ago

I did not find any workaround, except copy/paste the parent functions without the validate... call. My preference would be to only display a warning message when nifi or nifi-registry no accessible in valdate_parameters_versioning_support function, so failing elegantly.

andyadamides commented 2 years ago

In our environment too, nifi and nifi registry are not deployed on the same host, and I have been getting similar problems. I see the validate_parameters_versioning_support is trying to be called whenever I try to execute import_flow_version:

ApiException: (403)
Reason: Forbidden
HTTP response body: Unable to view system diagnostics. Contact the system administrator.

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
/var/folders/56/pz_5hgsx53v380x5hw57j190mtvx8p/T/ipykernel_62047/2730610096.py in <module>
     10                                 bucket.identifier,
     11                                 encoded_flow=json.dumps(flow),
---> 12                                 flow_name=flow_name)
     13     print(vflow)
     14     # time.sleep(5)

/usr/local/lib/python3.7/site-packages/nipyapi/versioning.py in import_flow_version(bucket_id, encoded_flow, file_path, flow_name, flow_id)
    690                          "name for a flow in this bucket, but not both")
    691     # Now write the new version
--> 692     nipyapi.utils.validate_parameters_versioning_support()
    693     return create_flow_version(
    694         flow=ver_flow,

/usr/local/lib/python3.7/site-packages/nipyapi/utils.py in validate_parameters_versioning_support()
    533 def validate_parameters_versioning_support():
    534     """Convenience method to check if Parameters are supported"""
--> 535     nifi_check = enforce_min_ver('1.10', bool_response=True)
    536     registry_check = enforce_min_ver(
    537         '0.6', service='registry', bool_response=True)

+++

It looks like it is related. Checking the Nifi Registry, I can see that the relevant flow becomes visible in the UI after executing import_flow_version, but it is shown with 0 versions which suggests the actual version was not created, where is also where the code fails above because it tries to validate_parameters_versioning_support first... and also, trying to list_flow_versions on the same imported flow returns me 404.

Trying out @mixam85 's suggestion worked, but that means would potentially need to copy/maintain the nipyapi codebase internally. Can someone suggest a workaround other than the one suggested above from @mixam85?

ChrisSamo632 commented 2 years ago

Just ran into this too

My suggestion would be to update validate_parameters_versioning_support in utils to accept a bool method parameter that indicates whether the check is being used by something that is using NiFi or just Registry (e.g. a registry_only flag that would be True for create_flow_version and import_flow_version but False for save_flow_ver

When this new flag is True that don't bother with the nifi_check call at all (just set that to False within the utils method); when the new flag is False then check both NiFi and Registry as present

Other than falling back to using NiFi Toolkit to import the Flow from the command line, I'm not sure of any workaround to this currently

Chaffelson / nipyapi