mqboard put needs to use flow-control

[palmtreefrb]

Any idea what might be happening here. Running /blinky/sync (venv) fredrick@fredrick-Studio-XPS-435MT:~/Development/micropython/mqboard/blinky$ ./sync.sh --server mqtt.xxxxxxx.com --port 8883 --tls --prefix aiotcomqtt Target directory /lib ok sntp.py ok sysinfo.py ok logging.py ok board.py ok mqtt.py ok mqrepl.py ok watchdog.py put shadiff ../mqtt_async/mqtt_async.py -> /lib/mqtt_async.py MQRepl protocol error put: message missing: 22 vs. 21 Aborted!

[tve]

The error states that message 21 got lost (it received message 22 when it expected 21). I have to admit I haven't seen that error triggered... mqboard doesn't use flow-control for put (only for ota), I wonder whether your MQTT server is configured to only buffer up to 20 messages per client or per topic... Is there a way you can check that? If that's the case, try to double the BUFLEN https://github.com/tve/mqboard/blob/master/mqboard/engine.py#L9, that will cut the number of messages in half...

[palmtreefrb]

Changed BUFLEN to 2400. Below is the output...

(venv) fredrick@fredrick-Studio-XPS-435MT:~/Development/micropython/mqboard/blinky$ ./sync --server mqtt.xxxxxx.com --port 8883 --tls --prefix aiotcomqtt --verbose --timeout 120 Target directory /lib Connecting to mqtt.xxxxxx.com:8883 Subscribing to aiotcomqtt/mqb/reply/out/syyIGwzl and aiotcomqtt/mqb/reply/err/syyIGwzl Pub aiotcomqtt/mqb/cmd/eval/syyIGwzl #0 last=True len=499 Received reply on topic 'aiotcomqtt/mqb/reply/out/syyIGwzl' with QoS 1 0.928kB in 1.427s -> 0.650kB/s ok sntp.py ok sysinfo.py ok logging.py ok board.py ok mqtt.py ok mqrepl.py ok watchdog.py put shadiff ./../mqtt_async/mqtt_async.py -> /lib/mqtt_async.py Subscribing to aiotcomqtt/mqb/reply/out/tEXJWKHS and aiotcomqtt/mqb/reply/err/tEXJWKHS Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #0 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #1 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #2 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #3 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #4 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #5 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #6 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #7 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #8 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #9 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #10 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #11 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/tEXJWKHS//lib/mqtt_async.py #12 last=True len=2040 Received reply on topic 'aiotcomqtt/mqb/reply/out/tEXJWKHS' with QoS 1 34.830kB in 4.652s -> 7.487kB/s Target directory /src Subscribing to aiotcomqtt/mqb/reply/out/onaw5ZgQ and aiotcomqtt/mqb/reply/err/onaw5ZgQ Pub aiotcomqtt/mqb/cmd/eval/onaw5ZgQ #0 last=True len=499 Traceback (most recent call last): File "/safemode/mqrepl.py", line 313, in _msg_cb File "/safemode/mqrepl.py", line 192, in _do_eval File "", line 18, in File "", line 17, in _ls File "", line 14, in _ls_dir OSError: [Errno 2] ENOENT mkdir /src Subscribing to aiotcomqtt/mqb/reply/out/MngltQmr and aiotcomqtt/mqb/reply/err/MngltQmr Pub aiotcomqtt/mqb/cmd/eval/MngltQmr #0 last=True len=135 Received reply on topic 'aiotcomqtt/mqb/reply/out/MngltQmr' with QoS 1 0.132kB in 0.611s -> 0.216kB/s put missing ./blinky.py -> /src/blinky.py Subscribing to aiotcomqtt/mqb/reply/out/zCZ8r7gV and aiotcomqtt/mqb/reply/err/zCZ8r7gV Pub aiotcomqtt/mqb/cmd/put/zCZ8r7gV//src/blinky.py #0 last=True len=1692 Received reply on topic 'aiotcomqtt/mqb/reply/out/zCZ8r7gV' with QoS 1 1.654kB in 0.613s -> 2.698kB/s Target directory / Subscribing to aiotcomqtt/mqb/reply/out/KWc2q5u8 and aiotcomqtt/mqb/reply/err/KWc2q5u8 Pub aiotcomqtt/mqb/cmd/eval/KWc2q5u8 #0 last=True len=496 Received reply on topic 'aiotcomqtt/mqb/reply/out/KWc2q5u8' with QoS 1 0.537kB in 0.614s -> 0.875kB/s put missing ../board/main.py -> /main.py Subscribing to aiotcomqtt/mqb/reply/out/HPG0-ImY and aiotcomqtt/mqb/reply/err/HPG0-ImY Pub aiotcomqtt/mqb/cmd/put/HPG0-ImY//main.py #0 last=False len=2802 Pub aiotcomqtt/mqb/cmd/put/HPG0-ImY//main.py #1 last=True len=846 Received reply on topic 'aiotcomqtt/mqb/reply/out/HPG0-ImY' with QoS 1 3.564kB in 0.434s -> 8.207kB/s put missing ./board_config.py -> /board_config.py Subscribing to aiotcomqtt/mqb/reply/out/gjR3kdNb and aiotcomqtt/mqb/reply/err/gjR3kdNb Pub aiotcomqtt/mqb/cmd/put/gjR3kdNb//board_config.py #0 last=True len=2702 Received reply on topic 'aiotcomqtt/mqb/reply/out/gjR3kdNb' with QoS 1 2.641kB in 0.799s -> 3.304kB/s ok boot.py

[palmtreefrb]

Uggg, just now finding out that my ESP32-DevKitC WROOM-32D does not have a user LED only a 5v LED

[tve]

Looks good! So the amount of buffering at the broker level most likely was the issue. This means I need to implement flow-control, just like I do for OTA (which is a bit on the hacky end). Grrr...