LocalHttpProxy connection crashes when trying to transfer larger payload.

cryptobench commented 1 year ago

Description

OS: Mac OSX

yagna daemon version: Yapapi master - 25th october 2022

Python version: 3.10.7

yapapi library version: 0.10.0.dev0

yapapi branch: Master

Description of the issue: When hosting a webserver on the provider via localhttpproxy and you try to upload a larger payload, then the VPN crashes or you receive aiohttp.client_exceptions.ClientOSError: [Errno 54] Connection reset by peer

Screenshots:

Steps To Reproduce

Launch a webserver with http-proxy and send a request to the webserver where the payload includes a file of size 10 MB ish

Expected behavior

Requests returns as expected

Logs and any additional context

Yagna

[2022-10-25T16:49:32.393+0200 WARN  ya_vpn::requestor] VPN WebSocket: VPN 959c80e218e24b498725a70b865a6ef5 connection timed out
[2022-10-25T16:49:32.393+0200 WARN  ya_vpn::requestor] VPN WebSocket: VPN 959c80e218e24b498725a70b865a6ef5 connection timed out
[2022-10-25T16:49:32.393+0200 WARN  ya_vpn::requestor] VPN WebSocket: VPN 959c80e218e24b498725a70b865a6ef5 connection timed out
[2022-10-25T16:49:32.393+0200 WARN  ya_vpn::requestor] VPN WebSocket: VPN 959c80e218e24b498725a70b865a6ef5 connection timed out

Yapapi

Error handling request
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/opt/homebrew/lib/python3.10/site-packages/yapapi/contrib/service/http_proxy.py", line 251, in _request_handler
    return await instance.handle_request(request)
  File "/opt/homebrew/lib/python3.10/site-packages/yapapi/contrib/service/http_proxy.py", line 153, in handle_request
    await ws.send_bytes(chunk)
  File "/opt/homebrew/lib/python3.10/site-packages/aiohttp/client_ws.py", line 156, in send_bytes
    await self._writer.send(data, binary=True, compress=compress)
  File "/opt/homebrew/lib/python3.10/site-packages/aiohttp/http_websocket.py", line 688, in send
    await self._send_frame(message, WSMsgType.BINARY, compress)
  File "/opt/homebrew/lib/python3.10/site-packages/aiohttp/http_websocket.py", line 659, in _send_frame
    await self.protocol._drain_helper()
  File "/opt/homebrew/lib/python3.10/site-packages/aiohttp/base_protocol.py", line 87, in _drain_helper
    await asyncio.shield(waiter)
aiohttp.client_exceptions.ClientOSError: [Errno 54] Connection reset by peer

cryptobench commented 1 year ago

Unable to recreate the crash now, but instead i'm receiving the following 90% of the time. Sometimes you are lucky to receive a "fast" provider that returns 200 and everything works.


[2022-10-25T22:11:58.647+0000 ERROR yapapi.contrib.service.http_proxy] Error receiving remote response. url=/service/create?deployment_id=15&build_path=dist, service=, exception=(expected string or bytes-like object) [Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/yapapi/contrib/service/http_proxy.py", line 159, in handle_request
    response = await response_parser.get_response()
  File "/usr/local/lib/python3.8/site-packages/yapapi/contrib/service/http_proxy.py", line 81, in get_response
    self.receive_data(ws_response.data)
  File "/usr/local/lib/python3.8/site-packages/yapapi/contrib/service/http_proxy.py", line 59, in receive_data
    parts = re.split(b"\r\n\r\n", data)
  File "/usr/local/lib/python3.8/re.py", line 231, in split
    return _compile(pattern, flags).split(string, maxsplit)
TypeError: expected string or bytes-like object
])

Error occurs here in yapapi https://github.com/golemfactory/yapapi/blob/888b25bcfe2d4ef6558b7e1a3b02c8a39d0beef1/yapapi/contrib/service/http_proxy.py#L165

shadeofblue commented 1 year ago

@cryptobench which yagna version are you using?

cryptobench commented 1 year ago

The one released during the community testing of the new network driver.

shadeofblue commented 1 year ago

@golemfactory/core I believe it has to do with the timeout on the websocket itself - could you confirm or deny that?

golemfactory / yapapi