Carglglz
commented
6 months ago @wohltat your test does not do what you think is doing e.g.
# allocate and prepare fixed block of data to be sent
b = bytearray(10_000)
for k in range(100):
p = k*100
b[p:p] = b'X'*96 + f'{k:3}' + '\n'
mv = memoryview(b)if you add
print(len(b), b[-100:])You will see that, 1) it has a length of 20_000 and 2) there are invalid UTF bytes.
Also here
...
await ws.send_str(b[0:100*n].decode())
n += 1
...this is sending an infinite growing string, so I'm not sure what you want to test with this...
anyway here is a proper test (working on unix and esp32 ports)
import aiohttp
import asyncio
import sys
import time
# allocate and prepare fixed block of data to be sent
b = bytearray(10_000)
for k in range(100):
p = k * 100
b[p : p + 100] = b"X" * 96 + f"{k:3}" + "\n"
mv = memoryview(b)
# print(len(b), b[-100:])
try:
size = int(sys.argv.pop(-1))
except Exception:
size = 2000
# sys.exit()
URL = "wss://<URL>:8448/echo"
sslctx = False
if URL.startswith("wss:"):
try:
import ssl
sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
sslctx.verify_mode = ssl.CERT_NONE
except Exception:
pass
async def ws_receive(ws: ClientWebSocketResponse):
try:
async for msg in ws:
if msg.type != aiohttp.WSMsgType.TEXT:
print("type:", msg.type, repr(msg.data))
except (TypeError, OSError) as e:
print("ws_receive", repr(e))
async def ws_test_echo(session):
n = 0
l = 0
m = 0
ws_receive_task = None
async with session.ws_connect(URL, ssl=sslctx) as ws:
ws_receive_task = asyncio.create_task(ws_receive(ws))
try:
while True:
t0 = time.ticks_ms()
while (n * 100) + size < len(b):
await ws.send_str(b[n * 100 : (n * 100) + size].decode())
dt = time.ticks_diff(time.ticks_ms(), t0) / 1e3
print(
"-------------------",
f"packet # {m} |",
f"total: {l/1000:.1f} KB |",
f"{(size/1000)/dt:.1f} KB/s |",
f"{((size*8)/dt)/1e3:.1f} kbps",
"------------------------",
end="\r",
)
n += 1
m += 1
l += size
t0 = time.ticks_ms()
await asyncio.sleep_ms(100)
n = 0
except KeyboardInterrupt:
pass
finally:
await ws.close()
async def main():
async with aiohttp.ClientSession() as session:
print(f"session test @ packet size: {size/1000:.1f} KB")
await ws_test_echo(session)
def run():
asyncio.run(main())
if __name__ == "__main__":
run()
and server
import ssl
from microdot import Microdot
from microdot.websocket import with_websocket
app = Microdot()
html = """<!DOCTYPE html>
<html>
<head>
<title>Microdot Example Page</title>
<meta charset="UTF-8">
</head>
<body>
<div>
<h1>Microdot Example Page</h1>
<p>Hello from Microdot!</p>
<p><a href="/shutdown">Click to shutdown the server</a></p>
</div>
</body>
</html>
"""
@app.route("/")
async def hello(request):
return html, 200, {"Content-Type": "text/html"}
@app.route("/echo")
@with_websocket
async def echo(request, ws):
t0 = time.ticks_ms()
l = 0
m = 0
while True:
data = await ws.receive()
size = len(data)
l += size
dt = time.ticks_diff(time.ticks_ms(), t0) / 1e3
print(
"-------------------",
f"packet # {m} |",
f"total: {l/1000:.1f} KB |",
f"{(size/1000)/dt:.1f} KB/s |",
f"{((size*8)/dt)/1e3:.1f} kbps",
"------------------------",
end="\r",
)
m += 1
t0 = time.ticks_ms()
# print(data[-4:].replace("\n", ""), end="\r")
await ws.send(data)
@app.route("/shutdown")
async def shutdown(request):
request.app.shutdown()
return "The server is shutting down..."
ssl_cert = "cert.pem"
ssl_key = "key.pem"
sslctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
sslctx.load_cert_chain(ssl_cert, ssl_key)
app.run(port=8448, debug=True, ssl=sslctx)
client
$ micropython ws_echo_test.py 2000
session test @ packet size: 2.0 KB
------------------- packet # 49 | total: 98.0 KB | 19.4 KB/s | 155.3 kbps -------------------------server
$ micropython ws_server_test.py
Starting async server on 0.0.0.0:8448...
------------------- packet # 570 | total: 1142.0 KB | 19.2 KB/s | 153.8 kbps ------------------------
wohltat
@Carglglz I have an application i want to send TLS / SSL packages over websockets using aiohttp on an ESP32. The problem is that the websockets fail after a short while when using packages that have a little bigger size around 2kB to 4kB.
Here is a simple test script:
remote
I'll get the following exception after a while. Just dealing with the exception is not satisfying since it takes a while and the applicaiton is blocked in that time.
(OSError: 113 = ECONNABORTED)
Noteworthy is that there are always some packages that micropyhon thinks are sent already but don't reach the other side. I also do not receive a websocket close package. This was tested on a remote Websocket server that i don't control.
local
When i try it on a local server, i see different problems.
(OSError: 104 = ECONNRESET)
The servers seems to not receive the complete message / only a corrupted message and closes the connection:
The closing of the websocket is also not handled properly by the micropython application / aiohttp. The close package is received but the connection is not closed automatically.
Another problem is that even if i deal with the exceptions and reconnect automatically, my application will eventually crash because of a run out of memory. This is really problematic to me since i only send messages of size 2k or so. I guess this is because of the not so memory economical design of aiohttp. For sending and receiving there is always a lot of new memory allocation involved. The use of preallocation and
memoryviewseems reasonable here.This is the local python websocketserver code:
MicroPython v1.23.0-preview.344.gb1ac266bb.dirty on 2024-04-29; Generic ESP32 module with ESP32