aida-ugent / SkillGPT

Other
56 stars 11 forks source link

Issues with Redis #3

Open Iago-bot opened 1 year ago

Iago-bot commented 1 year ago

Hello,

I have managed to execute the jupyter notebook api_requests.ipynb withouth any problem. However, when arriving to the cell "Label text", I get the following error when executing the response.post instruction:

ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read)) [1]

At the same time that this error is generated, the python -m api service stops, with an exception in ASGI application [2].

On the other way, when executing the cell "Initialize ESCO embeddings", a ConnectionResetError is produced when the request.post function is executed. [3]

I have accessed the Redis server using redis-cli, both from the host machine and from within the Docker container, and when executing the simple instruction "KEY *", the connection is refused and I get disconnected [4].

I am working on a server with 2 GPUs Nvidia GeForce.

Best regards

The traces of the abovementioned errors are:

[1]


ValueError Traceback (most recent call last) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in _update_chunk_length(self) 696 try: --> 697 self.chunk_left = int(line, 16) 698 except ValueError:

ValueError: invalid literal for int() with base 16: b''

During handling of the above exception, another exception occurred:

IncompleteRead Traceback (most recent call last) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in _error_catcher(self) 436 try: --> 437 yield 438

/opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in read_chunked(self, amt, decode_content) 763 while True: --> 764 self._update_chunk_length() 765 if self.chunk_left == 0:

/opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in _update_chunk_length(self) 700 self.close() --> 701 raise httplib.IncompleteRead(line) 702

IncompleteRead: IncompleteRead(0 bytes read)

During handling of the above exception, another exception occurred:

ProtocolError Traceback (most recent call last) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/models.py in generate() 750 try: --> 751 for chunk in self.raw.stream(chunk_size, decode_content=True): 752 yield chunk

/opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in stream(self, amt, decode_content) 571 if self.chunked and self.supports_chunked_reads(): --> 572 for line in self.read_chunked(amt, decode_content=decode_content): 573 yield line

/opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in read_chunked(self, amt, decode_content) 792 if self._original_response: --> 793 self._original_response.close() 794

/opt/anaconda3/envs/CPU/lib/python3.7/contextlib.py in exit(self, type, value, traceback) 129 try: --> 130 self.gen.throw(type, value, traceback) 131 except StopIteration as exc:

/opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/response.py in _error_catcher(self) 454 # This includes IncompleteRead. --> 455 raise ProtocolError("Connection broken: %r" % e, e) 456

ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

During handling of the above exception, another exception occurred:

ChunkedEncodingError Traceback (most recent call last)

in 18 } 19 response = requests.post(worker_addr + "/label_text", headers=headers, ---> 20 json=pload, stream=False) 21 22 print(response.text) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/api.py in post(url, data, json, **kwargs) 117 """ 118 --> 119 return request('post', url, data=data, json=json, **kwargs) 120 121 /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/api.py in request(method, url, **kwargs) 59 # cases, and look like a memory leak in others. 60 with sessions.Session() as session: ---> 61 return session.request(method=method, url=url, **kwargs) 62 63 /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json) 528 } 529 send_kwargs.update(settings) --> 530 resp = self.send(prep, **send_kwargs) 531 532 return resp /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/sessions.py in send(self, request, **kwargs) 681 682 if not stream: --> 683 r.content 684 685 return r /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/models.py in content(self) 827 self._content = None 828 else: --> 829 self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b'' 830 831 self._content_consumed = True /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/models.py in generate() 752 yield chunk 753 except ProtocolError as e: --> 754 raise ChunkedEncodingError(e) 755 except DecodeError as e: 756 raise ContentDecodingError(e) ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read)) ****************************************************************************************************************************************************************************************************************************************************************************************** [2] Traceback (most recent call last): File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/responses.py", line 277, in __call__ await wrap(partial(self.listen_for_disconnect, receive)) File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/responses.py", line 273, in wrap await func() File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/responses.py", line 250, in listen_for_disconnect message = await receive() File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 565, in receive await self.message_event.wait() File "/opt/anaconda3/envs/skillGPT/lib/python3.9/asyncio/locks.py", line 226, in wait await fut asyncio.exceptions.CancelledError: Cancelled by cancel scope 7fc6f80d8700 During handling of the above exception, another exception occurred: + Exception Group Traceback (most recent call last): | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 429, in run_asgi | result = await app( # type: ignore[func-returns-value] | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__ | return await self.app(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/fastapi/applications.py", line 276, in __call__ | await super().__call__(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__ | await self.middleware_stack(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__ | raise exc | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__ | await self.app(scope, receive, _send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ | raise exc | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ | await self.app(scope, receive, sender) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__ | raise e | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__ | await self.app(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__ | await route.handle(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle | await self.app(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/routing.py", line 69, in app | await response(scope, receive, send) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/responses.py", line 277, in __call__ | await wrap(partial(self.listen_for_disconnect, receive)) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 664, in __aexit__ | raise BaseExceptionGroup( | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) +-+---------------- 1 ---------------- | Traceback (most recent call last): | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/connection.py", line 865, in read_response | response = self._parser.read_response(disable_decoding=disable_decoding) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/connection.py", line 346, in read_response | result = self._read_response(disable_decoding=disable_decoding) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/connection.py", line 356, in _read_response | raw = self._buffer.readline() | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/connection.py", line 259, in readline | self._read_from_socket() | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/connection.py", line 209, in _read_from_socket | data = self._sock.recv(socket_read_size) | ConnectionResetError: [Errno 104] Connection reset by peer | | During handling of the above exception, another exception occurred: | | Traceback (most recent call last): | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/responses.py", line 273, in wrap | await func() | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/responses.py", line 262, in stream_response | async for chunk in self.body_iterator: | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/concurrency.py", line 63, in iterate_in_threadpool | yield await anyio.to_thread.run_sync(_next, iterator) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/anyio/to_thread.py", line 33, in run_sync | return await get_async_backend().run_sync_in_worker_thread( | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2106, in run_sync_in_worker_thread | return await future | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 833, in run | result = context.run(func, *args) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/starlette/concurrency.py", line 53, in _next | return next(iterator) | File "/home/iago/skillGPT/SkillGPT/skillgpt.py", line 169, in label_text_gate | res = self.label_embedding_redis(params) | File "/home/iago/skillGPT/SkillGPT/skillgpt.py", line 162, in label_embedding_redis | memory = RedisMemory(redis_host, redis_port) | File "/home/iago/skillGPT/SkillGPT/redisearch.py", line 51, in __init__ | existing_vec_num = self.redis.get(f'{esco_index}-vec_num') | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/commands/core.py", line 1790, in get | return self.execute_command("GET", name) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/client.py", line 1258, in execute_command | return conn.retry.call_with_retry( | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/retry.py", line 49, in call_with_retry | fail(error) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/client.py", line 1262, in | lambda error: self._disconnect_raise(conn, error), | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/client.py", line 1248, in _disconnect_raise | raise error | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/retry.py", line 46, in call_with_retry | return do() | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/client.py", line 1259, in | lambda: self._send_command_parse_response( | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/client.py", line 1235, in _send_command_parse_response | return self.parse_response(conn, command_name, **options) | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/client.py", line 1275, in parse_response | response = connection.read_response() | File "/opt/anaconda3/envs/skillGPT/lib/python3.9/site-packages/redis/connection.py", line 871, in read_response | raise ConnectionError( | redis.exceptions.ConnectionError: Error while reading from localhost:6379 : (104, 'Connection reset by peer') +------------------------------------ ****************************************************************************************************************************************************************************************************************************************************************************************** [3] --------------------------------------------------------------------------- ConnectionResetError Traceback (most recent call last) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 676 headers=headers, --> 677 chunked=chunked, 678 ) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 425 # Otherwise it looks like a bug in the code. --> 426 six.raise_from(e, None) 427 except (SocketTimeout, BaseSSLError, SocketError) as e: /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/packages/six.py in raise_from(value, from_value) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 420 try: --> 421 httplib_response = conn.getresponse() 422 except BaseException as e: /opt/anaconda3/envs/CPU/lib/python3.7/http/client.py in getresponse(self) 1343 try: -> 1344 response.begin() 1345 except ConnectionError: /opt/anaconda3/envs/CPU/lib/python3.7/http/client.py in begin(self) 305 while True: --> 306 version, status, reason = self._read_status() 307 if status != CONTINUE: /opt/anaconda3/envs/CPU/lib/python3.7/http/client.py in _read_status(self) 266 def _read_status(self): --> 267 line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") 268 if len(line) > _MAXLINE: /opt/anaconda3/envs/CPU/lib/python3.7/socket.py in readinto(self, b) 588 try: --> 589 return self._sock.recv_into(b) 590 except timeout: ConnectionResetError: [Errno 104] Connection reset by peer During handling of the above exception, another exception occurred: ProtocolError Traceback (most recent call last) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies) 448 retries=self.max_retries, --> 449 timeout=timeout 450 ) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 724 retries = retries.increment( --> 725 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] 726 ) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace) 402 if read is False or not self._is_method_retryable(method): --> 403 raise six.reraise(type(error), error, _stacktrace) 404 elif read is not None: /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/packages/six.py in reraise(tp, value, tb) 733 if value.__traceback__ is not tb: --> 734 raise value.with_traceback(tb) 735 raise value /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw) 676 headers=headers, --> 677 chunked=chunked, 678 ) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 425 # Otherwise it looks like a bug in the code. --> 426 six.raise_from(e, None) 427 except (SocketTimeout, BaseSSLError, SocketError) as e: /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/packages/six.py in raise_from(value, from_value) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw) 420 try: --> 421 httplib_response = conn.getresponse() 422 except BaseException as e: /opt/anaconda3/envs/CPU/lib/python3.7/http/client.py in getresponse(self) 1343 try: -> 1344 response.begin() 1345 except ConnectionError: /opt/anaconda3/envs/CPU/lib/python3.7/http/client.py in begin(self) 305 while True: --> 306 version, status, reason = self._read_status() 307 if status != CONTINUE: /opt/anaconda3/envs/CPU/lib/python3.7/http/client.py in _read_status(self) 266 def _read_status(self): --> 267 line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") 268 if len(line) > _MAXLINE: /opt/anaconda3/envs/CPU/lib/python3.7/socket.py in readinto(self, b) 588 try: --> 589 return self._sock.recv_into(b) 590 except timeout: ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) During handling of the above exception, another exception occurred: ConnectionError Traceback (most recent call last) in 6 } 7 response = requests.post(worker_addr + "/init_esco_embedding", headers=headers, ----> 8 json=pload) 9 10 print(response.content) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/api.py in post(url, data, json, **kwargs) 117 """ 118 --> 119 return request('post', url, data=data, json=json, **kwargs) 120 121 /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/api.py in request(method, url, **kwargs) 59 # cases, and look like a memory leak in others. 60 with sessions.Session() as session: ---> 61 return session.request(method=method, url=url, **kwargs) 62 63 /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json) 528 } 529 send_kwargs.update(settings) --> 530 resp = self.send(prep, **send_kwargs) 531 532 return resp /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/sessions.py in send(self, request, **kwargs) 641 642 # Send the request --> 643 r = adapter.send(request, **kwargs) 644 645 # Total elapsed time of the request (approximately) /opt/anaconda3/envs/CPU/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies) 496 497 except (ProtocolError, socket.error) as err: --> 498 raise ConnectionError(err, request=request) 499 500 except MaxRetryError as e: ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) ****************************************************************************************************************************************************************************************************************************************************************************************** [4] redis-cli 127.0.0.1:6379> KEY * Error: Connection reset by peer