search

sorgerlab / indra

INDRA (Integrated Network and Dynamical Reasoning Assembler) is an automated model assembly system interfacing with NLP systems and databases to collect knowledge, and through a process of assembly, produce causal graphs and dynamical models.
http://indra.bio
BSD 2-Clause "Simplified" License
177 stars 68 forks source link

INDRA REST API problems #1335

Closed lyfer233 closed 3 years ago

lyfer233 commented 3 years ago

I found these problems when I using the multiprocessing in Python to process some strings at trips module.

some log information as follows:

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

ERROR: [2021-10-18 05:09:41] indra.sources.trips.client - Problem with TRIPS query: status code 500

ERROR: [2021-10-18 05:09:41] indra.sources.trips.client - Problem with TRIPS query: status code 500

ERROR: [2021-10-18 05:09:41] indra.sources.trips.client - Problem with TRIPS query: status code 500

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

ERROR: [2021-10-18 05:09:41] indra.sources.trips.client - Problem with TRIPS query: status code 500

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

ERROR: [2021-10-18 05:09:41] indra.sources.trips.client - Problem with TRIPS query: status code 500

INFO: [2021-10-18 05:09:41] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:09:41] "POST /trips/process_text HTTP/1.1" 200 -

INFO: [2021-10-18 05:10:27] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:10:27] "POST /trips/process_text HTTP/1.1" 200 -

INFO: [2021-10-18 05:11:06] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:11:06] "POST /trips/process_text HTTP/1.1" 200 -

ERROR: [2021-10-18 05:11:13] api - Exception on /trips/process_text [POST]

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 174, in _new_conn

    conn = connection.create_connection(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 96, in create_connection

    raise err

  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 86, in create_connection

    sock.connect(sa)

TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen

    httplib_response = self._make_request(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 394, in _make_request

    conn.request(method, url, **httplib_request_kw)

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 239, in request

    super(HTTPConnection, self).request(method, url, body=body, headers=headers)

  File "/usr/lib/python3.8/http/client.py", line 1252, in request

    self._send_request(method, url, body, headers, encode_chunked)

  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request

    self.endheaders(body, encode_chunked=encode_chunked)

  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders

    self._send_output(message_body, encode_chunked=encode_chunked)

  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output

    self.send(msg)

  File "/usr/lib/python3.8/http/client.py", line 947, in send

    self.connect()

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 205, in connect

    conn = self._new_conn()

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 186, in _new_conn

    raise NewConnectionError(

urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7efe387ff8b0>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send

    resp = conn.urlopen(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen

    retries = retries.increment(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 574, in increment

    raise MaxRetryError(_pool, url, error or ResponseError(cause))

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='trips.ihmc.us', port=80): Max retries exceeded with url: /parser/cgi/drum?input=+Additionally%2C+Ink4c+loss+is+applied+to+the+standard+Ptch%2B%2F-+GEMM%2C+leading+to+a+modest+enhancement+of+phenotype+%28~45%25+incidence%29 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efe387ff8b0>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1950, in full_dispatch_request

    rv = self.dispatch_request()

  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1936, in dispatch_request

    return self.view_functions[rule.endpoint](**req.view_args)

  File "/usr/local/lib/python3.8/dist-packages/flask_restx/api.py", line 388, in wrapper

    resp = resource(*args, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 89, in view

    return self.dispatch_request(*args, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/flask_restx/resource.py", line 44, in dispatch_request

    resp = meth(*args, **kwargs)

  File "/sw/indra/rest_api/api.py", line 479, in post

    tp = trips.process_text(text)

  File "/sw/indra/indra/sources/trips/api.py", line 50, in process_text

    html = client.send_query(text, service_endpoint=service_endpoint,

  File "/sw/indra/indra/sources/trips/client.py", line 47, in send_query

    res = requests.get(url, query_args, timeout=3600)

  File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 75, in get

    return request('get', url, params=params, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 61, in request

    return session.request(method=method, url=url, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request

    resp = self.send(prep, **send_kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send

    r = adapter.send(request, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 516, in send

    raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='trips.ihmc.us', port=80): Max retries exceeded with url: /parser/cgi/drum?input=+Additionally%2C+Ink4c+loss+is+applied+to+the+standard+Ptch%2B%2F-+GEMM%2C+leading+to+a+modest+enhancement+of+phenotype+%28~45%25+incidence%29 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efe387ff8b0>: Failed to establish a new connection: [Errno 110] Connection timed out'))

INFO: [2021-10-18 05:11:13] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:11:13] "POST /trips/process_text HTTP/1.1" 500 -

INFO: [2021-10-18 05:11:27] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:11:27] "POST /trips/process_text HTTP/1.1" 200 -

ERROR: [2021-10-18 05:11:44] api - Exception on /trips/process_text [POST]

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 174, in _new_conn

    conn = connection.create_connection(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 96, in create_connection

    raise err

  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 86, in create_connection

    sock.connect(sa)

TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 699, in urlopen

    httplib_response = self._make_request(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 394, in _make_request

    conn.request(method, url, **httplib_request_kw)

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 239, in request

    super(HTTPConnection, self).request(method, url, body=body, headers=headers)

  File "/usr/lib/python3.8/http/client.py", line 1252, in request

    self._send_request(method, url, body, headers, encode_chunked)

  File "/usr/lib/python3.8/http/client.py", line 1298, in _send_request

    self.endheaders(body, encode_chunked=encode_chunked)

  File "/usr/lib/python3.8/http/client.py", line 1247, in endheaders

    self._send_output(message_body, encode_chunked=encode_chunked)

  File "/usr/lib/python3.8/http/client.py", line 1007, in _send_output

    self.send(msg)

  File "/usr/lib/python3.8/http/client.py", line 947, in send

    self.connect()

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 205, in connect

    conn = self._new_conn()

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 186, in _new_conn

    raise NewConnectionError(

urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7efe38725c70>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in send

    resp = conn.urlopen(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 755, in urlopen

    retries = retries.increment(

  File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 574, in increment

    raise MaxRetryError(_pool, url, error or ResponseError(cause))

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='trips.ihmc.us', port=80): Max retries exceeded with url: /parser/cgi/drum?input=+ITSN+regulates+endophilin+function+and+aids+in+recruitment+of+endophilin+A+to+sites+of+clathrin-mediated+SV+recycling%2C+allowing+vesicle+uncoating (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efe38725c70>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1950, in full_dispatch_request

    rv = self.dispatch_request()

  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1936, in dispatch_request

    return self.view_functions[rule.endpoint](**req.view_args)

  File "/usr/local/lib/python3.8/dist-packages/flask_restx/api.py", line 388, in wrapper

    resp = resource(*args, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 89, in view

    return self.dispatch_request(*args, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/flask_restx/resource.py", line 44, in dispatch_request

    resp = meth(*args, **kwargs)

  File "/sw/indra/rest_api/api.py", line 479, in post

    tp = trips.process_text(text)

  File "/sw/indra/indra/sources/trips/api.py", line 50, in process_text

    html = client.send_query(text, service_endpoint=service_endpoint,

  File "/sw/indra/indra/sources/trips/client.py", line 47, in send_query

    res = requests.get(url, query_args, timeout=3600)

  File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 75, in get

    return request('get', url, params=params, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 61, in request

    return session.request(method=method, url=url, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 542, in request

    resp = self.send(prep, **send_kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 655, in send

    r = adapter.send(request, **kwargs)

  File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 516, in send

    raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='trips.ihmc.us', port=80): Max retries exceeded with url: /parser/cgi/drum?input=+ITSN+regulates+endophilin+function+and+aids+in+recruitment+of+endophilin+A+to+sites+of+clathrin-mediated+SV+recycling%2C+allowing+vesicle+uncoating (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7efe38725c70>: Failed to establish a new connection: [Errno 110] Connection timed out'))

INFO: [2021-10-18 05:11:44] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:11:44] "POST /trips/process_text HTTP/1.1" 500 -

INFO: [2021-10-18 05:11:46] werkzeug - 172.17.0.1 - - [18/Oct/2021 05:11:46] "POST /trips/process_text HTTP/1.1" 200 -
lyfer233 commented 3 years ago

I ensure that I had looked through the INDRA docs, but I not find about "max length" or "max multiprocess / multithread" description.