[feature request] Proposing debounce filter

curlyel commented 3 years ago

Have you considered creating a third filter for debouncing readings beside mean and skip? I've checked your code and if I unterstand correctly,

mean - calculates a sliding average over windowSizenumber of readings before publishing skip - simply skips skipCount number of readings before publishing the next value

Quite useful from my point of view would be: debounce - basically quite similar to mean, BUT publishes immediately if the reading of a datapoint has changed remarkably (higher than a configured change threshold).

Example: When someone is interested in current power readings he/she likely don't want to publish every tiny change of e.g. 5 Watts to the mqtt broker which potentially triggers subsequent processing. (I guess this was the initial intention for your mean and skip filters ;-)

So in general, a reading update every minute or even longer might be sufficient usually.

BUT if some device get switched on drawing a hight amount of power, you may want to react immediately and not wait for the next minute's mqtt publish of the mean value.

So if the current power readings changes by more than the configured amount (e.g. by more than 500Watt) compared to the last published value, the power reading should be published immediately, probably by sending the mean value (which has been calculated til the big change so far) at first and then followed immediately by the current reading.

This could bring a good balance between fewer messages but still the possibility getting a quick response to big changes in current power.

Configuration probably:

        {
            "type": "Sml",
            "topic": "smartmeter/stromzähler",
            "device": "/dev/serial/by-id/usb-FTDI_FT230X_Basic_UART_D309TP6X-if00-port0",
            "baudrate": 9600,
            "MessageFilters": [
                {
                    "comment": "This is the power value of my smart meter",
                    "type": "debounce",
                    "datapoint": "1-0:16.7.0*255",
                    "windowSize": 60
                    "threshold": 500
                },
        }

oetken commented 3 years ago

@curlyel I made some changes to the mean filter to add you proposed behavior. Seems to work fine for me. I added an optional second parameter which basically reduces the window size to 1 after a threshold was reached and reincrements the window size by one each time the value sent counter is equal to the parameter until it is back to the original window size. Thus I get a higher resolution for a short period of time after a threshold was reached.

curlyel commented 3 years ago

Excellent!

The "window size slow start approach" after hitting the threshold is quite clever 👍 Better than what I had in mind ;-)

Many thanks for looking into it! I'm going to test this immediately, when back home...

oetken commented 3 years ago

Yep gives me nice detail on bigger steps and not too many data otherwise, which anyway is most of the time the case. Maybe there is still a better approach but I think it's fine for now.

Just want to do some integration test to verify that integrating my power still matches my energy and there is no mistake anywhere.

oetken / smartmetertomqtt

[feature request] Proposing debounce filter #3