Moving average

[baiti]

My understanding is the datagrams are sent by the device at a rate of 1 per second. Typically one wouldn't want to send an MQTT message once per second. For instance, I have configured my system such that it sends an MQTT message every 60s.

In such a configuration it would be very helpful if sml2mqtt could do a moving average over the "power" sensor. The moving average should be calculated based on the amount of samples that fit into one MQTT message interval. In my example where the device sends a datagram once every second and the MQTT message goes out every 30 seconds the moving average should be calculated over the last 30 samples periodically.

Could that be implemented? perhaps as a configurable option?

[spacemanspiff2007]

My understanding is the datagrams are sent by the device at a rate of 1 per second.

That's not correct. The datagrams are filtered by your custom filters. See docs for an example where there is a 5% change required for a report or a heartbeat is sent every 2 mins.

[baiti]

My "understanding" wasn't based on what sml2mqtt might be doing. Let me be more precise: At what frequency does the smart meter transmit datagrams over the serial interface?

P.S. I don't mean the baud rate. I mean the rate at which the smartmeter sends a full sml message.

[spacemanspiff2007]

At what frequency does the smart meter transmit datagrams over the serial interface?

Ca every second one datagram

[baiti]

Thanks for clarifying. Now what happens if I set:

    values:

      0100100700ff:
        mqtt:
          topic: power
        filters:
          - every: 60

What happens to the "other 59 readings" in between? I am setting every: 60 because a sampling rate of one message per minute is sufficient for my application. I don't need a one second resulution on my utility power meter, however, what I'd need is that the last every readings are summed up and averaged and the average (or median) is sent once per every to the mqtt broker.

How about a filter let's say: average or median so I could say:

    values:

      0100100700ff:
        mqtt:
          topic: power
        filters:
          - every: 60
          - median

From an implementation perspective, all that's needed would be a ring buffer also known as a circular buffer where the once per second readings go in and when it is time to construct an mqtt message, go read all those entries and determine the average or determine the median based on the filter that is selected in the config.yml.

Of course I could set my config to every 1 and subscribe to the mqtt topic and do the average or median myself or let that be "aggregated" by appropriate influxdb rules but that not only generates completely unnecessary network traffic, it also makes the processing in the influxdb ugly. It seems to me the most natural thing to handle this at the source, where the data is received and precessed at the sml level.

I am just not good enough with python to do a PR myself. What I could offer though is that I'd do some debug and testing work. I could even make changes to the code, if I knew the code flow better. I have looked at the sources, I know there is a device class and an mqtt class. I haven't found where the filters are and whether I'd be capable to implement a new filter for that but I will give it a try if you'd say you're too busy for instance. "Time" is not an issue for me.

[baiti]

Update: I have found src/sml2mqtt/sml_value/filter/xxx.py. This is obviously the place where filters go. I could think of two new filters:

• average
• median

... and taking all values from a ring buffer or circular buffer and doing the calculations is almost trivial. It looks like a use case of python's reduce functionality.

What I don't know is how to do the circular buffer and where to put it.

Of course there needs to be a change in:

                values={
                    'OBIS': SmlValueConfig(
                        mqtt=OptionalMqttPublishConfig(topic='OBIS'),
                        workarounds=[{'negative on energy meter status': True}],
                        transformations=[{'factor': 3}, {'offset': 100}, {'round': 2}],
                        filters=[{'diff': 10}, {'perc': 10}, {'every': 120}],
                    )
                }

could be a new transformation or a new filter there

[christiankratzer]

I have a small proof of concept for the moving average sitting in a pull request at: https://github.com/spacemanspiff2007/sml2mqtt/pull/32

[spacemanspiff2007]

Implemented in the rewrite