umqtt.simple's check_msg triggers OSError -1 with TLS servers

[SylvainGarrigues]

Please consider this code:

import utime
from umqtt.simple import MQTTClient

mqtt = MQTTClient("test_mqtt_client_id", "test.mosquitto.org",
                  port=8883, keepalive=30, ssl=True)

def on_topic_updated(topic, msg):
    print((topic, msg))

mqtt.set_callback(on_topic_updated)
mqtt.connect()
mqtt.subscribe(b"Topic/For/Sylvain")
while True:
    print("Checking msg...")
    mqtt.check_msg()
    utime.sleep(1)

It systematically gives the same output (and I can reproduce it with another broker e.g. AWS IoT)

$ /usr/local/Cellar/micropython/1.11/bin/micropython  main.py
Checking msg...
Checking msg...
Traceback (most recent call last):
  File "main.py", line 15, in <module>
  File "/Users/sylvain/.micropython/lib/umqtt/simple.py", line 204, in check_msg
  File "/Users/sylvain/.micropython/lib/umqtt/simple.py", line 173, in wait_msg
OSError: -1

What's strange is that if I use port 1883 and ssl=False (i.e. no encryption), the same code works.

I reproduced this issue on my Mac (MicroPython 1.11) and on my ESP8266 board (MicroPython 1.12).

Looping over mqtt.wait_msg() works like a charm, so I suspect switching back and forth a TLS socket from blocking to non-blocking generates such error.

[domenc]

Same here (ESP8266 and MicroPython 1.12). When change SSL to noSSL connection, works fine.

[samf48]

I too have this issue using a ublox modem and AWS over TLS. Are there any workarounds?

[sactre]

Same problem, in ESP32 it works perfectly but in ESP8266 it doesn't.

[CharlieAt]

there seems to be some kind of problem when the ssl socket is set to nonblocking and it occasionally returns b"" which results in the OSError(-1). Ignoring it did not help. to work around the problem I created a poll object after the ssl_wrap and in check_msg() I check the poll object instead of mucking around with setblocking(True/False).

YMMV / workaround / not an optimal solution, but it works for me

simple.py

def connect(self, clean_session=True):
    ...
    if self.ssl:
        import ussl
        self.sock = ussl.wrap_socket(self.sock, server_hostname=self.server)
    self.p = uselect.poll()
    self.p.register(self.sock, uselect.POLLIN)    

def check_msg(self):
    #self.sock.settimeout(0.0) #setblocking(False)
     res = self.p.ipoll(0)
     for s,e in res:
        if e & uselect.POLLIN: 
            return self.wait_msg()
     return None   

and comment out the #self.sock.setblocking(True) in get_msg()

[GadiHerman]

According to my test, the problem does not occur because of SSL. The problem is because the server's waiting time is over according to the keepalive=??? My solution is to add a timer that PINGs the server before the keepalive time expires. This solves the problem. Here is a sample code to solve the problem:

from machine import Pin, Timer
from umqtt.simple import MQTTClient
import ujson
import sys
import os

LED = Pin(2, Pin.OUT)
PING_PERIOD   = 120

CHANNEL_TOKEN = 'token_XXXXX'
CHANNEL_NAME  = 'esp32'
RESOURCE_NAME = 'led'
MQTT_SERVER = 'mqtt.beebotte.com'
MQTT_USER = 'token:' + CHANNEL_TOKEN
MQTT_TOPIC = CHANNEL_NAME + '/' + RESOURCE_NAME

def handleTimerInt(timer):
    client.ping()
    print('ping')

def callback_func(topic, msg):
    print("topic:",topic," msg:", msg)
    json_data= ujson.loads(msg)
    dt= json_data["data"]
    print("Data:"+str(dt))
    if str(dt) == 'True':
        LED.value(1)
    if str(dt) == 'False':
        LED.value(0)

# create a random MQTT clientID
random_num = int.from_bytes(os.urandom(3), 'little')
mqtt_client_id = bytes('client_'+str(random_num), 'utf-8')

client = MQTTClient(mqtt_client_id, MQTT_SERVER, user=MQTT_USER, password='', keepalive=PING_PERIOD*2  )

myTimer = Timer(0)

try:      
    client.connect()
    myTimer.init(period=PING_PERIOD*1000, mode=Timer.PERIODIC, callback=handleTimerInt)
except Exception as e:
    print('could not connect to MQTT server {}{}'.format(type(e).__name__, e))
    sys.exit()

client.set_callback(callback_func)
client.subscribe(MQTT_TOPIC)

while True:
    try:
        client.wait_msg()
    except KeyboardInterrupt:
        print('Ctrl-C pressed...exiting')
        client.disconnect()
        sys.exit()