search

Chaffelson / nipyapi

A convenient Python wrapper for Apache NiFi
Other
245 stars 76 forks source link

Parameter Contexts blocking export-import of versioned flows #229

Closed Chaffelson closed 3 years ago

Chaffelson commented 3 years ago

From NiFi Mailing list:

HI Team,

We are facing the below mentioned issue while deploying the Nifi flow from one Nifi Registry bucket to another bucket using Nipy API, if flow contains any parameter context. However, it is working fine if flow doesn’t contain any parameter context.

Issue is while calling nipyapi.versioning.deploy_flow_version API with following error message

raise ApiException(http_resp=r)

nipyapi.nifi.rest.ApiException: (500)

Reason: Internal Server Error

HTTP response headers: HTTPHeaderDict({'Date': 'Tue, 20 Oct 2020 07:55:30 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31540000', 'Content-Type': 'text/plain', 'Vary': 'Accept-Encoding', 'Content-Length': '79', 'Server': 'Jetty(9.4.19.v20190610)'})

HTTP response body: An unexpected error has occurred. Please check the logs for additional details.

There is no error/exception in any other additional logs.

It seems that there is some issue in exporting and importing the flow from source bucket to destination bucket using Nipy API. As after importing the flow in destination bucket, it is also giving error while importing it at destination Nifi instance using Nifi UI from destination bucket.

And it is working fine while importing the flow through destination Nifi UI from source bucket. But we are not getting any error in nipyapi.versioning.export_flow_version and nipyapi.versioning.import_flow_version API calls

Could you please look into it and suggest if it is not possible through Nipy API if flow contains any parameter context. Or is there any other way to do it.

I have attached the source code for the reference. We are using following versions: Docker Nifi – 1.11.4 Docker Nifi Registry – nifi-registry-0.7.0 Nipy API - 0.14.3

In order to properly test this we should add Parameter Context handling to NiPy, construct tests for it, then add tests to the versioning code for this case. I'll make a start on the Parameter contexts today

Chaffelson commented 3 years ago

Controls for Parameter Contexts added in Next branch, https://github.com/Chaffelson/nipyapi/blob/next/nipyapi/parameters.py

Will look into test case for this tomorrow

Chaffelson commented 3 years ago

Have started investigating, looks like I need to make a test harness for situations involving two separate NiFi/Registry environments so we can test import/export between them. This would also help us test for nested Process Groups and a few a other usecases that people have asked about.