thingsboard / thingsboard

Open-source IoT Platform - Device management, data collection, processing and visualization.
https://thingsboard.io
Apache License 2.0
17.23k stars 5.08k forks source link

[Feature Request] Redis rule node cache #8221

Open ShvaykaD opened 1 year ago

ShvaykaD commented 1 year ago

Issue

Rule nodes such as "delay" and "deduplication" use the "tellSelf" method to schedule delay or to perform the deduplication analysis. tellSelf method sends messages to the current Rule Node with a specified millisecond delay. The main issue of this method is that message that sends using it is not queued and may be lost in case of a server restart.

tellSelf method:

    /**
     * Sends message to the current Rule Node with specified delay in milliseconds.
     * Note: this message is not queued and may be lost in case of a server restart.
     *
     * @param msg
     */
    void tellSelf(TbMsg msg, long delayMs);

Solution

Solution: Add the ability to use Redis cache with Sets implementation to store delayed messages and messages used in deduplication analysis.

Additional context

Rule node cache service will be stored in TbContext interface and can be accessed using the following method:

    Optional<RuleNodeCacheManager> getRuleNodeCacheManager();

Interface methods:

public interface RuleNodeCacheManager {

    void add(String key, String value);

    void add(String key, EntityId id);

    void add(EntityId key, Integer partition, TbMsg value);

    void removeStringList(String key, List<String> values);

    void removeEntityIdList(String key, List<EntityId> values);

    void removeTbMsgList(EntityId key, Integer partition, List<TbMsg> values);

    Set<String> getStrings(String key);

    Set<EntityId> getEntityIds(String key);

    Set<TbMsg> getTbMsgs(EntityId key, Integer partition);

    void evictTbMsgs(EntityId key, Integer partition);

    void evict(String key);

}

NOTE: List of methods might be changed in the final version. Check the updates in the corresponding release branch after the official release.

ryan-2048 commented 11 months ago

Hi