search

bemasher / rtlamr-collect

Data aggregation for rtlamr.
GNU Affero General Public License v3.0
171 stars 29 forks source link

Add SCM message type #11

Closed KanyonKris closed 5 years ago

KanyonKris commented 6 years ago

My power meter outputs the SCM message type, please add support for SCM.

I assume that SCM is trickier because the last reading (with time stamp) must be saved and then subtracted from the next reading to get the power consumption between readings. Not too hard, but can be messy if meter ID(s) are not specified to keep the number of meters in check. I propose limiting SCM support to some low number of meter IDs, perhaps even just one.

I may take a crack at coding SCM support, but I'm a novice programmer and have no experience with Go.

Thanks for these programs. I've been logged my power usage for months now, but not very nicely and I'd like to a proper job of it with InfuxDB.

bemasher commented 6 years ago

The primary issue is that meters typically update their consumption values at a different rate than transmitting messages. This means it's not uncommon to receive a message with the exact same contents from a meter more than once. For example, a lot of SCM-capable power meters update consumption every 5 minutes, and transmit every 30 seconds.

At the moment rtlamr-collect is structured to only really handle IDM messages because they provide historical data and more importantly, an interval index. With the interval index you can determine whether or not a message is part of a new interval, the last one we are aware of, or if we missed some intervals in between.

I have considered adding logic to handle SCM's, but it wouldn't keep any history. Instead, it would only parse and forward the message directly to InfluxDB. InfluxDB is fairly efficient for doing calculations like derivatives, which would take care of not keeping any historical data in rtlamr-collect itself.

I'll take a look at what would be necessary to add that.

KanyonKris commented 5 years ago

@bemasher thank you for that detailed explanation. It explains why I was getting very noisy power consumption values using only simple math. Sounds like I could get some useful values if I used a longer time interval, like an hour - that way the uncertainty of when the reading occurred would not matter as much. But if you can come up with a way to use derivatives to get a pseudo real time reading (5-10 minute updates would be fine for me), that would be cool.

mash55 commented 5 years ago

@bemasher - First of all, thanks for the provided tools. After realizing that all my utility meters (gas, water, electricty (PV production and NET meter)) can be detected by your tool, I was hoping to graph all of them in Grafana. I am especially interested in the NET meter data. Unfortunately only the PV-production meter is sending out idm messages. Gas and NET meter are sending scm and the water meter is sending our r900 messages. Bottom line, it was very educational for me to 'play' with all the tools and I am hoping that you can and will implement support for other messages too.

bemasher commented 5 years ago

Support for all of the protocols rtlamr supports is coming, but will modify some of the current default behavior, so it will be a breaking change for anyone already using rtlamr-collect.

mash55 commented 5 years ago

looking forward to the new version - thanks

bemasher commented 5 years ago

rtlamr-collect has been significantly rewritten to more gracefully handle meters that report differential data like IDM and NetIDM, and now supports SCM, SCM+, R900 and R900 BCD.

I don't have the opportunity to test this with all of the different protocols, so please let me know if you find any more issues.

I will need to re-write some of the continuous queries I provided with init.iql as well.