Web Workflow Impacts MQTT message flow

[niccit]

CircuitPython version

Adafruit CircuitPython 9.0.2

Code/REPL

import wifi
import adafruit_connection_manager
import adafruit_minimqtt as my_mqtt

cert_file = mqtt_data["cert_file"]
with open(cert_file, 'r') as file:
    cert_data = file.read()

radio = wifi.radio
pool = adafruit_connection_manager.get_radio_socketpool(radio)
ssl_context = adafruit_connection_manager.get_radio_ssl_context(radio)
ssl_context.load_verify_locations(cadata=cert_data)

mqtt_client = my_mqtt.MQTT(
            broker=mqtt_data["server"],
            port=mqtt_data["port"],
            username=mqtt_data["username"],
            password=mqtt_data["key"],
            socket_pool=pool,
            ssl_context=ssl_context,
            is_ssl=true,
        )

def message(client, topic, message):
    # listen for messages from sensors being monitored
    if "pir1" in topic:
        log_message = "Motion detected by PIR sensor 1"
        mqtt_client.publish(topic, log_message)
        trip_zone(relay_pin)

def connected(client, userdata, flags, rc):
    my_mqtt.subscribe(topics)  # subscribe to topics

# Behavior when disconnected from MQTT broker
# Log a warning and try to reconnect
def disconnected(client, userdata, rc):
    mqtt_client.connect()

def trip_zone(pin):
    if pin.value is False:
        pin.value = True
        time.sleep(4)
        pin.value = False

# Set up MQTT for publish/subscribe
mqtt_client.on_message = message
mqtt_client.on_connect = connected
mqtt_client.on_disconnect = disconnected

# Subscribe to sensor feeds in MQTT
# All feeds should be in the data.py file
topics = home-monitoring.pir1-logging

while True:
   mqtt_client.loop(timeout=5)
   time.sleep(0.25)

Behavior

No error messages

Description

Once code is running:

1. In one window subscribe to the MQTT server on the topic in the code
2. In a second window send a message to the MQTT server on the topic in the code
3. Confirm message sent, received, and relay tripped - will all happen within 2 seconds
4. Now, open Web Workflow for the board (in this case a Feather ESP32 V2)
5. Send another message the MQTT server on the topic in the code Nothing happens, no message received
6. Refresh the Web Workflow page (reload button) Message will send, relay will trip - but relay will not reset
7. Refresh the Web Workflow page (reload button) Relay resets

Additional information

Other odd behaviors happen including:

1. DNS resolution for circuitpython.local will fail temporarily although you can access the WWF via IP
2. MQTT sending will not work reliably once the WWF is closed and the board will need to be reset

This also happens when I access the WWF for the sensor board (QTY ESP32-S3 - running 9.0.0) I saw the same behavior on the Feather board using 9.0.0

MQTT server is running locally (Eclipse Mosquitto on a Raspberry Pi 4)

[tannewt]

Does MQTT work ok when web workflow isn't enabled?

[niccit]

Yes, perfectly. My project had been running since Saturday without any weirdness. And being super clear, WWF is still enabled. I just haven't accessed since Saturday.

On Mon, Apr 1, 2024, 11:36 Scott Shawcroft @.***> wrote:

Does MQTT work ok when web workflow isn't enabled?

— Reply to this email directly, view it on GitHub https://github.com/adafruit/circuitpython/issues/9123#issuecomment-2030323059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKAPB5WKLBMOLQWE7LS4ATY3GSJJAVCNFSM6AAAAABFPXBPEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQGMZDGMBVHE . You are receiving this because you authored the thread.Message ID: @.***>

[niccit]

If you'd like a video or to see it over a zoom/hangout call I'm happy to do that.

20 years in QA and I know how important a picture can be with weird issues like this. :)

Code:

• Main Program https://github.com/niccit/home_system_zone
• Helper Classes https://github.com/niccit/circuitpy_helpers/tree/changes-to-timelord (branch: changes-to-timelord)

~nicci

On Mon, Apr 1, 2024 at 11:45 AM Nicci Tynen @.***> wrote:

Yes, perfectly. My project had been running since Saturday without any weirdness. And being super clear, WWF is still enabled. I just haven't accessed since Saturday.

On Mon, Apr 1, 2024, 11:36 Scott Shawcroft @.***> wrote:

Does MQTT work ok when web workflow isn't enabled?

— Reply to this email directly, view it on GitHub https://github.com/adafruit/circuitpython/issues/9123#issuecomment-2030323059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKAPB5WKLBMOLQWE7LS4ATY3GSJJAVCNFSM6AAAAABFPXBPEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQGMZDGMBVHE . You are receiving this because you authored the thread.Message ID: @.***>

[tannewt]

I totally believe that it's working this way. :-) Figuring out why is the hard part. We have to do some tricky socket monitoring in a separate task that pokes CircuitPython.

Is MQTT using UDP?

[niccit]

Running a netstat on the server - seeing only TCP for both ports 8883 and 1883

ntynen-pi@mosquitopi:~ $ netstat -ntuap (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:631 0.0.0.0: LISTEN - tcp 0 0 0.0.0.0:1883 0.0.0.0: LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN - tcp 0 0 0.0.0.0:8883 0.0.0.0: LISTEN - tcp 0 0 192.168.86.47:22 192.168.86.39:44132 ESTABLISHED - tcp 0 0 192.168.86.47:8883 192.168.86.42:59270 ESTABLISHED - tcp6 0 0 ::1:631 ::: LISTEN - tcp6 0 0 :::1883 ::: LISTEN - tcp6 0 0 :::22 ::: LISTEN - tcp6 0 0 :::8883 ::: LISTEN - udp 0 0 0.0.0.0:5353 0.0.0.0: - udp 0 0 192.168.86.47:68 192.168.86.1:67 ESTABLISHED - udp 0 0 0.0.0.0:33069 0.0.0.0: - udp 0 0 0.0.0.0:631 0.0.0.0: - udp6 0 0 :::5353 ::: - udp6 0 0 :::43050 :::*

[tannewt]

Ok, thanks for checking. Any idea if 8.x has the same issue?

[niccit]

Oddly I went to CP9 because I couldn't get it working with SSL and my MQTT server. Luckily though I backed up the board (yay esptool). I restored it and got it working with SSL and can say that no, using the WWF in CP8 does not display the same issue as CP9.

[tannewt]

Thanks for the info! CP8 -> CP9 is a pretty big change for networking. It's probably a tricky problem to run down. I'm working on other stuff at the moment but appreciate the detailed report.

[niccit]

If there's anything I can do to help just ask!

On Fri, Apr 5, 2024, 09:48 Scott Shawcroft @.***> wrote:

Thanks for the info! CP8 -> CP9 is a pretty big change for networking. It's probably a tricky problem to run down. I'm working on other stuff at the moment but appreciate the detailed report.

— Reply to this email directly, view it on GitHub https://github.com/adafruit/circuitpython/issues/9123#issuecomment-2040251257, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKAPB4YH76VP5HTS56I3UTY33IW3AVCNFSM6AAAAABFPXBPEGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQGI2TCMRVG4 . You are receiving this because you authored the thread.Message ID: @.***>

[dhalbert]

Debugging note: check tasks that are doing select on sockets