Open jsphuebner opened 11 months ago
jsphuebner
commented
11 months ago I got a reply from Tibber today that the ebz DD3 ODZ1 meter is definitely supported. So I hope to see data soon. A firmware update was installed on the Pulse IR today, version 1001-b6715572
Kristian-R
commented
11 months ago Hi Johannes, nice to hear that your meter is supported. If you still don't get any data make sure you did not "misconfigure" your pulse by setting it up as "1000 impulse/kWh" with your LED-experiment. Under the node-tab there is a configure-link. Usually it is set to autoprobe.
Concerning your issue with not blocking the meter with the pulse this project might be interesting: https://github.com/evcc-io/evcc/discussions/7070
And this ist what I use to receive the mqtt-data and ingest in home assistant https://github.com/micw/tibber-pulse-reader
By the way: if someone knows how to decode the SML-MQTT directly in home assistant - pleas leave a note.
Regards Kristian-R
jsphuebner
commented
11 months ago Yes I've set it back to autoprobe I also send the raw string read from the meter via MQTT so I can replicate it anywhere. I want it to work on the actual meter first before experimenting with that.
disaster123
commented
10 months ago Any news?
christiankratzer
commented
10 months ago Stumbled upon this here.
I have another variation of a script that pulls data from the bridge via http and publishes decoded values over mqtt.
https://github.com/christiankratzer/pulse2mqtt
I use this to poll the tibber bridge every 10s for the latest value. I do get the occasional corrupted sml packet from the bridge that I throw away.
Also you could poll every second using the http interface but would have frequent collisions with duplicate data. I use the transactionid to dedup.
jsphuebner
commented
10 months ago Finally got the pulse to like my eBZ :) It has to sit on the top port, the front port doesn't work. Anyway, got the stream and it is super simple. Topic: $aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish/TJH01/yyy/obis_stream yyy in this case is the eui (see metric) It just contains the raw data from the meter, in my case
/EBZ5DD3BZ06ETA_107
1-0:0.0.0*255(1EBZzzzz)
1-0:96.1.0*255(1EBZzzzz)
1-0:1.8.0*255(009094.27916715*kWh)
1-0:16.7.0*255(000169.25*W)
1-0:36.7.0*255(000133.51*W)
1-0:56.7.0*255(000035.74*W)
1-0:76.7.0*255(000000.00*W)
1-0:32.7.0*255(233.4*V)
1-0:52.7.0*255(234.5*V)
1-0:72.7.0*255(234.6*V)
1-0:96.5.0*255(001C0104)
0-0:96.8.0*255(08F0EAB3)
!The Tibber app shows my current power consumption. I'll now try to generate the message myself along with the metrics and see how that goes
jsphuebner
commented
10 months ago The metrics now look like this. IR reader (TFD01) sends every 5 minutes:
{
"$type" : "node_status",
"node_status" : {
"product_id" : 49344,
"bootloader_version" : 17563650,
"meter_mode" : 4,
"node_battery_voltage" : 3.435,
"node_temperature" : 28.166,
"node_avg_rssi" : -43.557,
"node_avg_lqi" : 193.705,
"radio_tx_power" : 0,
"node_uptime_ms" : 2435139570,
"meter_msg_count_sent" : 99,
"meter_pkg_count_sent" : 133,
"time_in_em0_ms" : 4351,
"time_in_em1_ms" : 33,
"time_in_em2_ms" : 295767,
"acmp_rx_autolevel_300" : 146,
"acmp_rx_autolevel_9600" : 139
},
"hub_attachments" : {
"meter_pkg_count_recv" : 133,
"meter_reading_count_recv" : 99,
"node_version" : "1001-b6715572"
}
}So now meter_pkg_count_recv and meter_reading_count_recv are >0
And the wifi bridge TJH01 sends every 2 minutes:
{
"$type" : "status",
"device_id" : "yyy",
"ts" : 1692217927,
"status" : {
"$type" : "metrics",
"ssid" : "tach",
"rssi" : -73,
"bssid" : "3481c4e31c3f",
"channel" : 6,
"mqttcon" : 0,
"wificon" : 0,
"id" : "yyy",
"uptime" : 1771259,
"efr_uptime" : 1771152,
"vcc" : 3.49,
"vacrms" : 227.22,
"heap" : 124808,
"coredump_available" : true,
"model" : "TJH01",
"nodes" : [ {
"$type" : "node",
"id" : 1,
"eui" : "dc8e95fffeb82bfe",
"model" : "TFD01",
"version" : "1001-b6715572",
"rssi" : -64.5310287475586,
"seen" : 2,
"available" : true,
"pubcnt" : 93,
"ota_state" : "idle",
"ota_status" : "up2date",
"manifest_version" : "1001-b6715572",
"up2date" : true
} ],
"ota" : [ {
"model" : "tibber-pulse-ir-hub-esp32",
"manifest_version" : "1357-05c0c2f6",
"current_version" : "1357-05c0c2f6",
"state" : "up2date"
}, {
"model" : "tibber-pulse-ir-hub-css",
"manifest_version" : "1357-05c0c2f6",
"current_version" : "1357-05c0c2f6",
"state" : "up2date"
}, {
"model" : "tibber-pulse-ir-hub-js",
"manifest_version" : "1357-05c0c2f6",
"current_version" : "1357-05c0c2f6",
"state" : "up2date"
}, {
"model" : "tibber-pulse-ir-hub-efr32-fg23",
"manifest_version" : "584-9be5169b",
"current_version" : "584-9be5169b",
"state" : "up2date"
}, {
"model" : "tibber-pulse-ir-hub-efr32-fg1",
"manifest_version" : "584-9be5169b",
"current_version" : "584-9be5169b",
"state" : "up2date"
} ],
"hub_ota_status" : "up2date",
"manifest_state" : "synced",
"manifest_version" : "2306.4",
"manifest_group" : "",
"netflags" : 25,
"ip" : "192.168.188.31",
"gateway" : "192.168.188.1",
"dns0" : "192.168.188.1",
"dns1" : "0.0.0.0",
"dns2" : "0.0.0.0",
"dns3" : "0.0.0.0",
"dns4" : "0.0.0.0",
"dns5" : "0.0.0.0",
"dns6" : "0.0.0.0",
"dns7" : "0.0.0.0",
"dns8" : "0.0.0.0",
"mac" : "34:94:54:5e:84:0",
"ap_mac" : "34:94:54:5e:84:1"
}
}Here pubcnt > 0
I reckon the various uptimes should also be plausible. They are in 1s resolution.
jsphuebner
commented
10 months ago I have successfully sent data to tibber! I found the topic you need to send to contains the eui, dc8e95fffeb82bfe in my case. So the full topic is $aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish/TFD01/dc8e95fffeb82bfe/obis_stream
Due to my guerilla solar I have negative power and tibber doesn't like that. But I don't want to modify the meter string. I wonder if the original pulse somehow handles this. It is very unreliable, if it just moves by 1mm the data is already destroyed, i.e. you only get part of the string.
UPDATE: never mind this. While in the Apps home screen the data seems to freeze on negative values, they are being correctly recorded in the historic data
jsphuebner
commented
10 months ago And more updates. After like 15 minutes the joy was over. The data froze and the icon disappeared in the app. I plugged in the original bridge and the icon reappeared. I'm thinking there is some kind of hand shake that I failed to do
Update2: it is now running stable since 20h. In one metric I still send "yyy" as id, that was not appreciated ;)
jsphuebner
commented
10 months ago I have published my code and instructions here: https://github.com/jsphuebner/esp-egycounter/tree/main/battery-control/tibber (tibbersend.py is in parent directory)
reinhard-brandstaedter
commented
2 months ago Hi @jsphuebner I went through your code and tried to replicate that tibbersender setup. I do have my own ESP IR reader built which is now decoding SML (extracting current power value) and reporting to MQTT. In my journey to replace the tibber pulse IR header I wanted to use your tibbersender.py script to read the raw SML from MQTT and then send it to tibber's MQTT, along with the other metrics/heartbeats.
However whatever I try that script won't connect successfully to tibber's MQTT (double checked all settings).
I feel there is some extra added check on tibber side to identify real devices. At one point my real pulse stopped working - well it worked according to dmseg command on console - but no data was shown in the App.
Tibber support told me that suddenly there were two Pulse devices registered on their side and they had to remove one to get it back working.
Have you experienced anything like this? Maybe there is an additional check on what the pulse reports to tibber during pairing/first setup? Maybe any additional (hardware) info that is cross-checked?
I also tried having the pulse report to my local MQTT but it seems that clearing the ca_cert via console isn't successful and replacing it with my own CA seems to always truncate/garbage the cert value stored on the bridge...
I have now built a hardware proxy that is palced between the smartmeter IR interface and the pulse. It is an ESP that reads the IR signal via a photodiode and puts out plain SML via an IR led (which the pulse then receives). At the same time it also decodes SML and reports current power for my own usage. Works but that is not really a elegant solution...
jsphuebner
commented
2 months ago I never had that issue. I did go through the entire setup procedure in the Tibber app and verified the original pulse was (somewhat) working. Then I switched over to tibbersender You may also need to check the content of the static json files (metric and event) and get them close to what your bridge produces.
Indeed, once you have removed the cert info it is impossible to get it back in (probably by resetting the bridge and starting over).
reinhard-brandstaedter
commented
2 months ago Interesting, I tried to connect to tibber's MQTT broker with all the details (CA, Cert, key), client ID. And I don't even get a successful connection with mqtt client (cancels even before any subscription)...
christiankratzer
commented
2 months ago Who said you can subscribe to the topics on their broker. I assume you can only publish with those credentials. Also you should not have the pulse connected and try to connect at the same time with the same clientId.
If people keep messing with their broker without knowing what they are doing this will lead to tibber eventually closing this up for everybody.
Just let the pulse submit and consume from the pulses http server.
reinhard-brandstaedter
commented
2 months ago @christiankratzer First: a MQTT connection (establishing the SSL connection) should work. As I wrote I'm not even subscribing to anything nor publishing, just connecting. Second: I know that the pulse is only publishing SML data and the app is subscribing to read the updates that have been processed on tibber side. Third: I'm well aware that the same clientid is only allowed once, but that is irrelevant for the SSL connection, plus the pulse was disconnected at the same time Fourth: consuming via API from the tibber http server is a rather problematic solution if you want to control anyting in realtime locally or if various other things already have been set up in a home automation scenario. A few things in my household depend on the data from the smartmeter which I have configured before the tibber pulse and I'd like to keep them that way, so the pulse is for me just another consumer/reporter and should not be a replacement for the data collection.
I understand your concern regarding "messing" with the broker. But I also gave tibber that feedback that they should offically support the custom submit of data (without the need of an expensive, battery-operated IR reader) for people that are already getting the data in another way. (like it is possible in Norway i.e).
I have sort of misused issue #1 for discussing the pulse data format. The goal is to use your own meter reading setup and "spoof" tibber into thinking you're sending data from the Pulse. This is useful if your meter is not supported by tibber or the port is already blocked by your own reader. Some say the Pulses data rate is not usable for running zero export controllers.
Thought it might be a good idea to recollect this in a new issue. At first @Kristian-R found you need to empty the encryption keys by nulling the respective parameters, e.g.
param_set ca_cert \Then we got some messages. The topic pattern is $aws/rules/ingest_tibber_bridge_data/tibber-bridge/xxx/publish/TJH01/yyy where xxx is a long "Tibber ID" and yyy is the device ID. There are two device IDs, one for the bridge, one for the Pulse IR. So far we have seen 3 topics:An event message looks like this:
A metric dump looks like this (Bridge):
or that (Pulse IR):
And finally an impulse message looks like this:
{"$type": "imp_data", "timestamp_ms": 34652489,"delta_ms": 11782,"kw":0.030555, "kwh": 0.8297}A test run with pulses generated from the actual meter power yielded no display in the app though. Maybe tibber has disabled this method. So if someone could intercept real data from a meter talking either in the binary SML or the ASCII IEC-62056-21 format that would greatly help in writing a tibber export script