Messages droped without error message

[equidna]

I'm using this python module to send messages to a MQTT broker in Home Assistant. The code regarding the MQTT client is the following:

topic = '/homeassistant/zone_clusters'
client = mqtt_client.Client(client_id)
client.username_pw_set(username, password)
client.connect(broker, port)

time.sleep(row_time)
client.publish_qos = 2
for _, row in zone_clusters_df.iterrows():
    json_row = row.to_json()
    (rc, mid) = client.publish(topic=topic, payload=json_row, qos=2)
    if rc != 0:
        print(rc + ' - ' + mid)
    client.loop()
    time.sleep(row_time)
client.disconnect()
print('zone_clusters - done')`

But some of the messages are dropped without warning. I can see that by counting the number of messages received by listening to the MQTT topic (using node red).

Am I doing anything wrong in the code? If I'm using qos 2, shouldn't I receive at least a warning about a dropped message? Thank you.

[realtimeprojects]

I observed a similar problem using "qos=1":

When publishing multiple messages with qos=1 over single connetion, only the first 20 messages are delivered to the broker. Beginning from the 21st message, no more messages are delivered to the broker. This is 100% reproducible with paho mqtt client v1.6.1 and qos=1.

When using qos=0, there is no issue.

[MattBrittan]

As per the docs:

It is strongly recommended that you use loop_start(), or loop_forever(), or if you are using an external event loop using loop_read(), loop_write(), and loop_misc(). Using loop() on it's own is no longer recommended.

If you just want to send a bunch of messages and wait until they have been completely sent then use publish.multiple (or see it's source for the suggested technique).

@realtimeprojects my guess would be that you are not calling a loop function at all (the devil is in the detail).

Note that many brokers impose limits on the number of messages in flight; for instance Mosquitto defaults to 20 (see the max_inflight_messages setting) - if this limit is exceeded then messages are dropped. This is a common issue; QOS2 messages take a while to process (the publish is sent, a PUBREC is recieved, a PUBREL is sent then a PUBCOMP is received) and if you send messages quickly you can easily exceed the 20 messages in flight (just due to network latency).

Hopefully the above helps; if not I would need considerably more information (the first step is to confirm where the issue is; as mentioned above it may well be the broker that is dropping messages).