Switch to an event based architecture

maecki-maecki commented 3 years ago

I know it is a heavyweight task, but I guess the platform would in the long term benefit from an event-based architecture replacing the timer/pull based architecture it uses now.

Advantages I see:

strict decoupling of all modules like Inputs/Outputs/Functions/Influx Feeder/UI which do not need anything else to see but the event bus
lowest possible latency by event triggers as callbacks for changed values
no more need to tune time intervals for different operations
easy integration to other products by bridging to (or even native internal use of) an external message bus/broker (like mqtt)
easier development of future modules & drivers and thus a better support for broader installation base and community
scalability to large amount of input/output/data with high throughput

Disadvantages I see:

lot of work, almost every module would be affected (except probably timeseries access and maybe configuration/storage of modules)
it would be a complete redesign which would brake almost any existing customizations
almost certainly needs a long-running fork or new project so there might be a long timespan where two versions are worked on
UI would probably have to be split into a ‚Designer‘ where you can setup and configure the system and a ‚Runtime‘ where data is displayed and interaction is possible (I see this also as an advantage as it makes maintainability better and access more robust)

What d‘you think?

kizniche commented 3 years ago

Thanks for the proposal. I'm always interested in considering ways to improve the system. Could you help me understand a bit more with an example of how and where this would be used in Mycodo?

maecki-maecki commented 3 years ago

This would allow for an asynchronous yet very low-latency communication between modules (e.g. Reactive Pattern).

Controller          Broker/Bus          Input            Output 1
   |-—-subsc. Input 1-—>|                 |                 |
   |                    |<——subsc. for switch on messages-——|
   |                    |<— pub. measure —|                 |
   |<—-notify measure—-—|                 |                 |
   o Calculate
   |—-publish Out 1 on—>|                 |                 |
   |                    |————-notify do switch on ———-———-—>|

No one needs polling, everyone has a messagehandler that is called when a certain condition is met (there are also patterns if mutliple input data or criteria are needed). This makes each participant rather simple in interface and also testable …

Think of every object having ‘onMessage’ Handlers like in mqtt that are called whenever something relevant for the object is happening.

This could be either tightly integrated with something like mqtt or can be achieved solely within python e.g. with RxPY or some other framework(s) …

P.S.: This would also allow a pattern that in Zigbee is called ‘binding’ that makes an input directly trigger an output without any controller e.g. for a simple switch on/off operation

maecki-maecki commented 3 years ago

example for a mqtt based event-driven architecture: https://brewblox.netlify.app/

kizniche / Mycodo

Switch to an event based architecture #1050