Open-source, scalable, and fault-tolerant MQTT broker able to handle 4M+ concurrent client connections, supporting at least 3M messages per second throughput per single cluster node with low latency delivery. The cluster mode supports more than 100M concurrently connected clients.
This PR migrates the storage of device persisted messages and session context from PostgreSQL to Redis, optimizing data management while introducing Redis as an additional third-party dependency alongside PostgreSQL.
Details:
Migration to Redis: The data previously stored in the following PostgreSQL tables has been moved to Redis:
device_publish_msg: Stored information about device messages, including client ID, serial number, topic, time, packet ID, payload, and other properties.
device_session_ctx: Stored session context for devices, including the last updated time, last serial number, and last packet ID.
Creation of Lua Scripts: Developed Lua scripts to manage the saving of device persisted messages and device session context in Redis. These scripts ensure that the saving process is atomic, preventing inconsistencies and improving data integrity.
Automatic Cleanup in Redis: Previously, periodic jobs were scheduled to clean up device persisted messages. With the Redis implementation, this cleanup now happens automatically as part of the Lua scripts, reducing the need for additional scheduled jobs.
Third-Party Dependency Changes: This PR introduces Redis as a required third-party dependency specifically for storing device persisted messages and session context. PostgreSQL will continue to be used in other areas of the project, but Redis will now be an integral component for this specific functionality.
Pull Request description
Summary:
This PR migrates the storage of device persisted messages and session context from PostgreSQL to Redis, optimizing data management while introducing Redis as an additional third-party dependency alongside PostgreSQL.
Details:
General checklist
Back-End feature checklist