History data

[NigelCoxon]

Hi Serge

Firstly thanks for implementing this software. My system has been running very well for a while now, and I am able to look at my 3 phase power consumption via Openhab. I would be very interested to see the Current Cost history data as well. I could probably implement rules in Openhab to integrate the power readings and generate the energy used over a period, but it would be much cleaner to use the history data emitted by the Envi device.

At first I would just be interested in the most recent history items i.e. h002, d001, m001 and would use Openhab's persistence tools to store and graph these items. In future I might also want the rest of the history. If the Current Cost Forwarder published all the history data with appropriate topics, users could simply subscribe to the history items they were interested in.

Hopefully this would not be too difficult to implement.

Regards Nigel

[SR-G]

Hey. Can you please check the https://github.com/SR-G/current-cost-forwarder/blob/master/src/test/java/org/tensin/ccf/model/message-hist.xml example and tell me if you have the same output with your multiple channels (and post somehere an example if you have one) ? (you're speaking of "d001" / "m001" that i don't have with only one channel). I'll look into this.

[SR-G]

Hey again. I made a first implementation but i'm not very happy with it. As i don't use history messages and don't have any use cases for them, i really don't see how to map / transform XML history messages to output topics and how to use them with openhab.

Right now you may have in the code (assuming m001 > month, d001 > daily, h001 > hourly) (variables ${sensor}, ${seed}, ${type} are optional) :

• metrics/current-cost/${sensor}/history/daily/
• metrics/current-cost/${sensor}/history/monthly/
• metrics/current-cost/${sensor}/history/hourly/

But i don't know what to do with the history "seed" (the '578' in 'm578'). You can put it in the topic name but i really don't think that it makes sense.

Problem is also that XML history messages are splitted by the current cost device iteself : multiple history message are sent each ~10 secs and, as far as i can tell after a quick look, each of them only contains a few lines (the 4 last lines), without delimiters between XML messages, so it's not immediate to rebuild the whole history to send it as a consolidated message (or even to send only the "freshest" value). I really don't know how to deal with all that in openhab (how to consolidate datas, as i suppose that the same 'm578' value will be found in several history messages in the end).

If you have any idea feel free to share ;)

[NigelCoxon]

Hi

I have captured some data from my Envi CC128 which covers 2 dumps of the history data. I have edited out some of the "watt" items in the time between the history dumps. Hopefully you will be able to see it here https://drive.google.com/file/d/0B_bBOKRzv3VPVkhMNXNUZUdNcVk/view?usp=sharing

I agree that it is difficult to see how best to use all this data. For me, in Openhab, simply using the most recent hourly, daily, monthly readings will be good. I will then persist these and use openhab charting to display the history as it builds up over time. I do not think it is worth trying to use all the historical data output by the CC128.

There are some further issues to sort out though:

• According to the current cost documentation here http://www.currentcost.com/cc128/xml.htm (my unit reports v0.11 protocol) , the CC128 should transmit the most recent kwhr data with the tag h002. Looking at my data, the last hourly data is h004. This may be a temporary glitch, or maybe a bug in my unit.

  (edit - I thought it should emit an h002 tag, but it doesnt actually say that in the documentation. Looking at Dale lane's blog http://dalelane.co.uk/blog/?p=456, his xml also shows the most recent tag as h004 - his v0.11 xml is at the bottom of the post) So it looks like the units are consistent).

• I wasn't sure if the daily data was the last 24 hours worth, or genuinely the last day from midnight to midnight. It appears to be the latter, since it has not changed between the 2 history outputs. The monthly data has also not changed. However, the CC128 clock loses quite a lot of time when the history is output, so I suspect the daily figures will change after more than 24 hours, and may not be very accurate. Also I do not think the CC128 has an internal calendar, so I suspect the monthly data probably represents 30 days worth, rather than a calendar month.
• in my data, the most recent monthly (m001, m002,m003) readings are negative, but older ones are not. I can't find any mention of this in the documentation

So to conclude, I will be very happy if I can access the most recent history readings ie h002 (or h004 in my case), d001 and m001. In the early days, it would be good to keep an eye on the next few older items, to watch how often they change, and to check the maths (do the last 12 2 hourly readings add up to the day figure)

For the Current Cost Forwarder software, there needs to be a way to specify which readings to forward. I would suggest a simple list e.g. "h002,h004,d001,d002,m001,m002,m003". these could then be extracted from the history data, and published to MQTT with exactly these tags. They could be seperated into Hourly/Daily/Monthly topics according to the first letter. I do not think there is a need to aggregate any of the readings. If for some strange reason someone wants to look at history item h476, they could do so.

Cheers

[NigelCoxon]

Here's a thought I used to use Dale Lane's Current Cost Gui (CCGUI) (https://code.google.com/p/currentcostgui/) to chart my CC128 data. Unfortunately my Windows 8 laptop no longer recognises the rather old USB serial adaptor that I use for my CC128, so I havent used it for ages. There is an option in Dale's code to load the data via MQTT and then chart the data in various ways. The python MQTT client he uses is only available to IBM employees, but it should be possible to amend the code to use another library (eg Eclipse Paho MQTT Python client library).

I can envisage a setup where Current Cost Forwarder (CCF) publishes live and history data to MQTT. Openhab and CCGUI subscribe to this. Openhab will integrate the data into its environment, while CCGUI has some powerful charing options.

I will investigate the MQTT topics/format that CCGUI expects

[SR-G]

Thanks for the informations and the link to the specification.

As i've never looked into those history messages, i thought (wrongly) that the most recent values were the highest one (whereas h002 > 2 hours ago, and so on).

So in fact, what i've commited yesterday should already work, you just have to add : --broker-topic-history 'metrics/current-cost/history/${sensor}/${type}/${seed}' and subscribe only to the topic you need (h002 or h004), for example for the whole house : metrics/current-cost/history/0/hourly/h002

The only thing is that at this time, i fill every topic (h002, h004, ... h578), which is useless if you want just the last history, but should not be a problem (without subscriptions on all those topics, the MQTT broker will of course discard those values).

[NigelCoxon]

Hi

I haven't had a chance to look at any of this for a while now. I would like to try your latest version (as per your last commit) but I dont have a development environment setup. Could you please build another release so I can try it. Many thanks

[SR-G]

Hey, I think the 1.0.1-SNAPSHOT was in fact up to date (generated on 17 march). Anyway i've just re-uploaded my last distribution zip, still including the last evolutions for your use case (but github still see the release 1.0.1-SNASHOT a bit older, until i release a 1.0.1 version). Just grab https://github.com/SR-G/current-cost-forwarder/releases/download/1.0.1-SNAPSHOT/current-cost-forwarder-1.0.1-SNAPSHOT.zip