search

aws-samples / shopfloor-connectivity

Shop Floor Connectivity (SFC) is an industrial data ingestion enabler, that can quickly deliver customizable greenfield & brownfield connectivity solutions.
https://aws.amazon.com/blogs/industries/collecting-data-from-industrial-devices-to-aws-services/
MIT No Attribution
27 stars 4 forks source link

MQTT-Adapter only fetches last message on interval #25

Open fwerkmeister opened 16 hours ago

fwerkmeister commented 16 hours ago

We are using the MQTT-Adapter to fetch messages from an EMQX broker deployed with AWS Greengrass. We have multiple clients that send messages to the broker. We are processing the messages using shopfloor-connectivity-framework and forward them to IOT-Core using the AWS-IOT-Core Target Type.

With a higher number of message we recognised that not all messages are arriving on IOT-Core. We discovered that this is due to the Interval set within the schedule. Obviously only the last message is fetched when the interval timer fires. Also setting the interval to a minimum of couple 100 milliseconds, there are still messages that get lost.

For our use case it is crucial that all messages arriving at the MQTT broker are fetched by the shopfloor-connectivity-framework. It would be a good improvement for us to adapt the MQTT-Adapter so that no messages get lost.

Currently we are using version 1.4.1

asamerh4 commented 16 hours ago

Thanks Felix for raising that - we're looking into that asap.

arieleeuw commented 10 hours ago

Currently the adapter acts in keep-last mode, we can add an option to switch to keep-all mode in which case the received values over a read interval are returned as a list value.