Greengrass publish_to_topic unreliable?

[wills721]

Describe the bug MQTT from Greengrass appears to succeed but message never makes it to AWS

To Reproduce call ipc_client.publish_to_topic

Expected behavior It flows to AWS

Actual behavior This is my current code to publish a message over MQTT to AWS with a Retry.

def publish_binary_message_to_topic(ipc_client, topic, message): try: msg_string = json.dumps(message) binary_message = BinaryMessage(message=bytes(msg_string, 'utf-8')) publish_message = PublishMessage(binary_message=binary_message) except: log_debug_message('Exception occurred when publishing the event: data={data}')

max_retries = 5
retry_interval = 1  # seconds
retries = 0
while retries < max_retries:
    try:
        ipc_client.publish_to_topic(topic=topic, publish_message=publish_message)
        log_debug_message(f"PublishBinaryMessage SUCCESS Tries: {retries}")
        return True
    except Exception as e:
        log_debug_message(f"PublishBinaryMessage Failed. Attempt {retries + 1} of {max_retries}. Error: {e}")
        retries += 1
        time.sleep(retry_interval)

It is my understanding this is how to handle errors with this call. Plus I have a retry. HOWEVER, I am seeing rare cases where the result is SUCCESS from this call BUT in fact the message does not make it to AWS. I know this as I have a email subscriber on the Topic in AWS and I see every MQTT message that shows up. This appears to be a random thing.

Can anyone help? Is this a known bug?

Environment

• Nucleus version:
• MQTT Bridge version: 2.3.2
• Moquette version: 2.3.7
• GG version: /greengrass/v2, version=2.12.6

Additional context Add any other context about the problem here.

E.g. what is the impact of the bug? Greengrass is Unreliable!

[MikeDombo]

Hi, Publish to topic is only a local publish, the messages do not go to AWS. If you want to send an mqtt message you need to use publish to IoT core.

Publishing to IoT core will store messages in memory while offline unless you configure the disk spooler component.

[wills721]

Well, this does go to AWS but my config is: "mqttTopicMapping": { "LocalPubMessage-1": { "topic": "mytopic", "source": "Pubsub", "target": "IotCore" },

[MikeDombo]

You're using the MQTT bridge? You should not use the bridge for this because you already have IPC. Use the direct IPC calls: https://docs.aws.amazon.com/greengrass/v2/developerguide/ipc-iot-core-mqtt.html

[wills721]

Hmmm. Well, I'm not the original author of this code but I believe the original author was after off line (no internet) support/buffering and this was his approach. I can tell you this DOES work -- other than the occasional failures I mention in this original post.

[MikeDombo]

publish_to_iot_core also works offline and will be more reliable compared to the MQTT Bridge for you when you add the disk spooler as well. https://docs.aws.amazon.com/greengrass/v2/developerguide/disk-spooler-component.html

[wills721]

So you are saying if I were to do it this way I would still have offline and perhaps my issue would go away? I would like to test this. Do you have a sample DiskSpooler configuration I can see?

[MikeDombo]

Yes your issue will go away because the IPC call will tell you if it fails. If the call succeeded, then the message was persisted to the spooler.

https://docs.aws.amazon.com/greengrass/v2/developerguide/configure-greengrass-core-v2.html#configure-mqtt

[wills721]

I am going to work on testing this over the next couple days. One more question -- right now I have the Moquette and Mqtt broker as components in my custom component deployment-- sounds like your approach I would not need them and would only need Nucleus (which you always need anyway)?

[MikeDombo]

You do not need Moquette at all if you have no local MQTT client devices; even with your current approach. And yes, you can remove MQTT Bridge too

[wills721]

Thank you!