Chaffelson / nipyapi

A convenient Python wrapper for Apache NiFi
Other
246 stars 76 forks source link

nipyapi.canvas.list_all_input_ports(pg_id='root', descendants=False) fails with socket timeout error #150

Closed automationfirst closed 5 years ago

automationfirst commented 5 years ago

Nipyapi version: 0.13 NiFi version: nifi-1.8.0 NiFi-Registry version: NA Python version: Python 3.7.4 Operating System: Windows 2008 Server

Description

I was trying to get the list of all the input ports deployed onto canvas and it is getting failed with the below traceback errors.. Initially i thought cpu usage is high have checked the processor usage on the Nifi cluster and the core load average is about 2 on the 8 core nifi cluster.

We do have number of input ports deployed onto canvas. The rough count could be about 300 to 500.. But why it is getting timedout, I could not understand..

Is there anyway where we can change the timeout value? And is there a better way to get the list of all the input ports deployed onto canvas ?

What I Did

Paste the command(s) you ran and the output.
input_ports = nipyapi.canvas.list_all_input_ports(pg_id='root', descendants=False) If there was a crash, please include the traceback here.

Traceback (most recent call last): File "update_remote_input_ports_canvas.py", line 175, in input_ports = nipyapi.canvas.list_all_input_ports(pg_id='root', descendants=False) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\canvas.py", line 1339, in list_all_input_ports return list_all_by_kind('input_ports', pg_id, descendants) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\canvas.py", line 1333, in list_all_by_kind out += call_function(pg.id).getattribute(kind) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\apis\process_groups_api.py", line 1638, in get_input_ports (data) = self.get_input_ports_with_http_info(id, **kwargs) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\apis\process_groups_api.py", line 1719, in get_input_ports_with_http_info collection_formats=collection_formats) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\api_client.py", line 326, in call_api _return_http_data_only, collection_formats, _preload_content, _request_timeout) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\api_client.py", line 153, in __call_api _request_timeout=_request_timeout) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\api_client.py", line 349, in request headers=headers) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\rest.py", line 233, in GET query_params=query_params) File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\nipyapi\nifi\rest.py", line 224, in request raise ApiException(http_resp=r) nipyapi.nifi.rest.ApiException: (500) Reason: Internal Server Error HTTP response headers: HTTPHeaderDict({'Date': 'Mon, 16 Sep 2019 10:54:14 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'Content-Type': 'application/json' , 'Vary': 'Accept-Encoding', 'Content-Length': '40', 'Server': 'Jetty(9.4.11.v20180605)'}) HTTP response body: java.net.SocketTimeoutException: timeout

Urgency

We are in the process of testing the deployment. Without this we will not know how Nifi would behave on production. So it is kind of urgent