mqtt_as: How to catch exceptions when subscribtion callback throws one?

[mirko]

I'm using the mqtt_as MQTT implementation and doing my first uasyncio steps with the provided tutorial - thanks for both a lot btw!

I ended up with a very simple what I'd call mainloop in the end of my main script:

async def main_loop():                                                          
    while True:                                                                                        
        await asyncio.sleep(0)

asyncio.run(main_loop())

However wen I now send MQTT messages to topics I previously subscribed to - which throw exceptions - I really don't know how to handle them. When purposely publishing a message where my subscription callback function would throw an exception, I get:

From my script I get:

INFO:core:MQTT topic subscription callback called
DEBUG:core:  MQTT topic:   X/Y/Z
DEBUG:core:  MQTT payload: b'foobar'
ERROR:core:MQTT (sub)topic unknown -> throw MqttTopicException)
Task exception wasn't retrieved     <<<<<------------------------
future: <Task> coro= <generator object '_handle_msg' at 7f5b569e2920>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "mqtt_as/mqtt_as.py", line 546, in _handle_msg
  File "mqtt_as/mqtt_as.py", line 439, in wait_msg
  File "upyot/lib/Mqtt.py", line 364, in _cb_mqtt_sub
  File "upyot/lib/Mqtt.py", line 344, in _cb_mqtt_sub
MqttTopicException: UNKNOWN_TOPIC

Wrapping await asyncio.sleep(0) with try/except doesn't catch it either. So how to do? Sorry if this is a trivial question, maybe this might be one also addressed in the tutorial, e.g. under "5. Exceptions"?

[peterhinch]

As you have pointed out, this is covered in the tutorial section 5.

[mirko]

Thanks for your reply! Then maybe I'm misunderstanding:

Reading "During development it is often best if untrapped exceptions stop the program rather than merely halting a single task." it sounds like: a) it's only meant for development and not to deal and/or recover from thrown exceptions in production environments b) it says "stop[ping] the program rather than merely halting a single task", which reads to me like it's not what I'm trying to achieve. My goal is to handle certain exceptions expected or potentially not being expected in production environment, but not to halt/stop anything, but to continue as far as possible (e.g. by discarding an MQTT message being sent on an unsupported topic). I'm still unsure what would be the best-practice way to do so.

[peterhinch]

My point about development is this. A common bug is the case where an exception occurs which the designer has failed to anticipate. In the absence of a global exception handler a typical consequence is that the task with the error stops and a traceback appears at the REPL. However the rest of the application continues to run, typically issuing routine debug messages. The crucial traceback soon scrolls off screen. The outcome is that it is not obvious that a crucial failure has occurred. I find it helpful in development if untrapped exceptions stop the system dead.

It is of course desirable to trap and handle exceptions locally. The issue is how best to deal with the case where, in development, you haven't yet perfected this.

I don't think a global exception handler has much use in a production system. If it were ever triggered, what action could the code take? It might allow a watchdog to reset the system, but this is decidedly hacky. But your experience or views may differ on this.