Wrong data on KNX bus for Wh measurements

peterphilipp commented 8 months ago

PROBLEM DESCRIPTION

Hello,

I am trying to send "Energy Total" to the KNX Bus with my NOUS A1T plug.

Looking at the KNX specs "Wh" are supposed to be sent with the datatype 13.010 (which is 4 x 8-bit 2's complement -> https://support.knx.org/hc/en-us/articles/115001133744-Datapoint-Type).

But sending that measure to a group address with that data type produces completely wrong numbers on the KNX BUS. Only when I change the group address to 32-bit float data type like 14.056 for "W" the numbers are correct, but are not shown with the correct unit for example in Home-Assistant.

Is this a bug or by design? How can I change the data type that is sent to the bus in Tasmota. I cannot find anything in the documentation. Thanks for your help, great project!

Peter

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

[x] Read the Contributing Guide and Policy and the Code of Conduct
[x] Searched the problem in issues
[x] Searched the problem in discussions
[x] Searched the problem in the docs
[x] Searched the problem in the chat
[x] Device used (e.g., Sonoff Basic): NOUS A1T plug
[x] Tasmota binary firmware version number used: Tasmota 13.4.0(knx)
- [x] Pre-compiled
- [ ] Self-compiled
[x] Flashing tools used: Over Air

[x] Provide the output of command: Backlog Template; Module; GPIO 255:


Configuration output here:
08:42:27.888 CMD: Backlog Template; Module; GPIO 255
08:42:27.953 RSL: RESULT = {"NAME":"NOUS A1T","GPIO":[32,0,0,0,2720,2656,0,0,2624,320,224,0,0,0],"FLAG":0,"BASE":49}
08:42:28.161 RSL: RESULT = {"Module":{"0":"NOUS A1T"}}
08:42:28.368 RSL: RESULT = {"GPIO0":{"32":"Button1"},"GPIO1":{"0":"None"},"GPIO2":{"0":"None"},"GPIO3":{"0":"None"},"GPIO4":{"2720":"BL0937 CF"},"GPIO5":{"2656":"HLWBL CF1"},"GPIO9":{"0":"None"},"GPIO10":{"0":"None"},"GPIO12":{"2624":"HLWBL SEL_i"},"GPIO13":{"320":"Led_i1"},"GPIO14":{"224":"Relay1"},"GPIO15":{"0":"None"},"GPIO16":{"0":"None"},"GPIO17":{"0":"None"}}

- [x] If using rules, provide the output of this command: `Backlog Rule1; Rule2; Rule3`:
```lua
  Rules output here:
08:43:11.162 CMD: Backlog Rule1; Rule2; Rule3
08:43:11.241 RSL: RESULT = {"Rule1":{"State":"OFF","Once":"OFF","StopOnError":"OFF","Length":0,"Free":511,"Rules":""}}
08:43:11.450 RSL: RESULT = {"Rule2":{"State":"OFF","Once":"OFF","StopOnError":"OFF","Length":0,"Free":511,"Rules":""}}
08:43:11.700 RSL: RESULT = {"Rule3":{"State":"OFF","Once":"OFF","StopOnError":"OFF","Length":0,"Free":511,"Rules":""}}

[x] Provide the output of this command: Status 0:


STATUS 0 output here:
08:43:38.438 CMD: Status 0
08:43:38.446 RSL: STATUS = {"Status":{"Module":0,"DeviceName":"Tasmota Waschmaschine","FriendlyName":["Tasmota Waschmaschine"],"Topic":"tasmota_8328CF","ButtonTopic":"0","Power":1,"PowerOnState":3,"LedState":1,"LedMask":"FFFF","SaveData":1,"SaveState":1,"SwitchTopic":"0","SwitchMode":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"ButtonRetain":0,"SwitchRetain":0,"SensorRetain":0,"PowerRetain":0,"InfoRetain":0,"StateRetain":0,"StatusRetain":0}}
08:43:38.451 RSL: STATUS1 = {"StatusPRM":{"Baudrate":115200,"SerialConfig":"8N1","GroupTopic":"tasmotas","OtaUrl":"http://ota.tasmota.com/tasmota/release/tasmota-knx.bin.gz","RestartReason":"Software/System restart","Uptime":"48T13:54:02","StartupUTC":"2024-01-11T17:49:36","Sleep":50,"CfgHolder":4617,"BootCount":21,"BCResetTime":"2023-08-16T16:18:15","SaveCount":411,"SaveAddress":"FA000"}}
08:43:38.455 RSL: STATUS2 = {"StatusFWR":{"Version":"13.3.0(knx)","BuildDateTime":"2023-12-12T14:28:56","Boot":31,"Core":"2_7_4_9","SDK":"2.2.2-dev(38a443e)","CpuFrequency":80,"Hardware":"ESP8266EX","CR":"368/699"}}
08:43:38.459 RSL: STATUS3 = {"StatusLOG":{"SerialLog":2,"WebLog":2,"MqttLog":0,"SysLog":0,"LogHost":"","LogPort":514,"SSId":["WLAN Things",""],"TelePeriod":30,"Resolution":"558180C0","SetOption":["02008009","2805C80001000600003C5A0A192800000000","00000080","00006000","00004000","00000000"]}}
08:43:38.470 RSL: STATUS4 = {"StatusMEM":{"ProgramSize":632,"Free":368,"Heap":21,"ProgramFlashSize":1024,"FlashSize":1024,"FlashChipId":"1460C4","FlashFrequency":40,"FlashMode":"DOUT","Features":["0809","9F9AC587","04268011","000000CF","010013C0","8000F181","00004004","00001000","54000020","00000080","00000000"],"Drivers":"1,2,3,4,5,6,7,8,9,10,11,12,16,18,19,22,24,26,27,30,35,37,45,62,68","Sensors":"1,2,3,4,5,6","I2CDriver":"7"}}
08:43:38.476 RSL: STATUS5 = {"StatusNET":{"Hostname":"tasmota-8328CF-2255","IPAddress":"192.168.40.33","Gateway":"192.168.40.1","Subnetmask":"255.255.255.0","DNSServer1":"192.168.10.6","DNSServer2":"0.0.0.0","Mac":"34:94:54:83:28:CF","Webserver":2,"HTTP_API":1,"WifiConfig":4,"WifiPower":17.0}}
08:43:38.480 RSL: STATUS6 = {"StatusMQT":{"MqttHost":"","MqttPort":1883,"MqttClientMask":"DVES_%06X","MqttClient":"DVES_8328CF","MqttUser":"DVES_USER","MqttCount":0,"MAX_PACKET_SIZE":1200,"KEEPALIVE":30,"SOCKET_TIMEOUT":4}}
08:43:38.486 RSL: STATUS7 = {"StatusTIM":{"UTC":"2024-02-29T07:43:38","Local":"2024-02-29T08:43:38","StartDST":"2024-03-31T02:00:00","EndDST":"2024-10-27T03:00:00","Timezone":"+01:00","Sunrise":"07:33","Sunset":"18:31"}}
08:43:38.489 RSL: STATUS9 = {"StatusPTH":{"PowerDelta":[0,0,0],"PowerLow":0,"PowerHigh":0,"VoltageLow":0,"VoltageHigh":0,"CurrentLow":0,"CurrentHigh":0}}
08:43:38.497 RSL: STATUS10 = {"StatusSNS":{"Time":"2024-02-29T08:43:38","ENERGY":{"TotalStartTime":"2023-08-16T16:18:15","Total":99.826,"Yesterday":0.021,"Today":0.650,"Power":2650,"ApparentPower":2866,"ReactivePower":1093,"Factor":0.92,"Voltage":297,"Current":9.641}}}
08:43:38.502 RSL: STATUS11 = {"StatusSTS":{"Time":"2024-02-29T08:43:38","Uptime":"48T13:54:02","UptimeSec":4197242,"Heap":21,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":0,"POWER":"ON","Wifi":{"AP":1,"SSId":"WLAN Things","BSSId":"8A:2A:A8:84:2D:F6","Channel":1,"Mode":"11n","RSSI":66,"Signal":-67,"LinkCount":38,"Downtime":"0T01:18:24"}}}

- [x] Set `weblog` to 4 and then, when you experience your issue, provide the output of the Console log:
```lua
  Console output here:
09:54:54.424 CMD: weblog 4
09:54:54.429 RSL: RESULT = {"WebLog":4}
09:54:55.001 HLW: EPC 58, CFlen 18388 usec
09:54:55.328 CFG: Saved to flash at F9, Count 416, Bytes 4096
09:54:56.001 HLW: EPC 57, CFlen 16179 usec
09:54:57.001 HLW: EPC 57, CFlen 19570 usec
09:54:58.001 HLW: EPC 59, CFlen 16326 usec
09:54:59.001 HLW: EPC 57, CFlen 13284 usec
09:55:00.001 HLW: EPC 61, CFlen 22413 usec
09:55:01.001 HLW: EPC 54, CFlen 17003 usec
09:55:02.001 HLW: EPC 60, CFlen 18552 usec
09:55:03.001 HLW: EPC 51, CFlen 15258 usec
09:55:03.544 WIF: Checking connection...
09:55:04.001 HLW: EPC 53, CFlen 16865 usec
09:55:05.001 HLW: EPC 57, CFlen 16196 usec
09:55:06.001 HLW: EPC 52, CFlen 15842 usec
09:55:07.001 HLW: EPC 55, CFlen 17715 usec
09:55:08.001 HLW: EPC 50, CFlen 14509 usec
09:55:09.001 HLW: EPC 54, CFlen 17094 usec
09:55:10.001 HLW: EPC 53, CFlen 25114 usec
09:55:11.001 HLW: EPC 53, CFlen 19095 usec
09:55:12.001 HLW: EPC 54, CFlen 14953 usec
09:55:13.001 HLW: EPC 49, CFlen 18836 usec
09:55:14.001 HLW: EPC 51, CFlen 25869 usec
09:55:15.001 HLW: EPC 45, CFlen 116942 usec
09:55:16.001 HLW: EPC 4, CFlen 205090 usec
09:55:17.001 HLW: EPC 5, CFlen 197507 usec
09:55:18.001 HLW: EPC 5, CFlen 197369 usec
09:55:19.001 HLW: EPC 22, CFlen 8922 usec
09:55:19.288 RSL: STATE = {"Time":"2024-02-29T09:55:19","Uptime":"0T00:19:40","UptimeSec":1180,"Heap":22,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":25,"MqttCount":0,"POWER":"ON","Wifi":{"AP":1,"SSId":"WLAN Things","BSSId":"8A:2A:A8:84:2D:F6","Channel":1,"Mode":"11n","RSSI":66,"Signal":-67,"LinkCount":1,"Downtime":"0T00:00:05"}}
09:55:19.297 KNX: Power sent to 2/4/3
09:55:19.301 KNX: Energy Total sent to 2/4/4
09:55:19.305 RSL: SENSOR = {"Time":"2024-02-29T09:55:19","ENERGY":{"TotalStartTime":"2023-08-16T16:18:15","Total":100.056,"Yesterday":0.021,"Today":0.880,"Period":1,"Power":62,"ApparentPower":62,"ReactivePower":0,"Factor":1.00,"Voltage":306,"Current":0.202}}
09:55:20.001 HLW: EPC 97, CFlen 18643 usec

TO REPRODUCE

Steps to reproduce the behavior: Looking at the KNX specs "Wh" are supposed to be sent with the datatype 13.010 (which is 4 x 8-bit 2's complement -> https://support.knx.org/hc/en-us/articles/115001133744-Datapoint-Type).

Sending Wh that measure to a group address 13.010 with that data type produces completely wrong numbers on the KNX BUS. Only when I change the group address to 32-bit float data type like 14.056 for "W" the numbers are correct, but are not shown with the correct unit for example in Home-Assistant.

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen. Show correct Numbers on the KNX Bus for Wh with the datatype 13.010.

SCREENSHOTS

If applicable, add screenshots to help explain your problem. Screenshot for Data Type 14.056 in ETS grafik

Screenshot for Data Type 13.010 in ETS grafik

ADDITIONAL CONTEXT

Add any other context about the problem here.

(Please, remember to close the issue when the problem has been addressed)

github-actions[bot] commented 7 months ago

This issue has been automatically marked as stale because it hasn't any activity in last few weeks. It will be closed if no further activity occurs. Thank you for your contributions.

peterphilipp commented 7 months ago

Hi, it would be great if someone could look into that issue...

barbudor commented 7 months ago

Hi I looked at the code and I can confirm that all energy related measures are sent as 4-bytes float (Voltage, Current, Power, PowerFactor, as well as Energy Yesterday, Daily, Total)

I see also in your screen shot that when float datapoint type is selected, you get the correct values which is expected given the above comment.

What I don't get is where you get that from : "Wh" are supposed to be sent with the datatype 13.010" Can you please point to the proper specifications ?

barbudor commented 7 months ago

Ok, so this document seems to confirm what you are saying: https://support.knx.org/hc/en-us/articles/15392604906514-Interworking-Datapoint-types

@arendst it looks like that in order to be compliant with the standard, answer for Energy yesterday, Daily, Total should be changed from knx.answer_4byte_float() to knx.answer_4byte_int() with appropriate scaling to provide in Wh instead of kWh in order no to loose resolution

What do you think ? I can do the PR but I have no way of testing

arendst commented 7 months ago

Thx for diving in. Yes, no problem with a PR.

Noschvie commented 7 months ago

... if needed I can perform some tests this weekend.

barbudor commented 7 months ago

Hi @peterphilipp @Noschvie I'll prépare a test version tonight or thursday

barbudor commented 7 months ago

Does anyone knows about simple (and free) KNX debug tool that would display KNXnet/IP datagram ? Something like MQTT explorer but for KNX ? Doesn't need to be GUI, command line is fine too May be Wireshark has a KNX décoder ?

sfromis commented 7 months ago

I know nothing here, except that Google can lead to interesting results, like searching for wireshark knx. The page about Wireshark Display Filter Reference mentions filters knx and kip, depending on before/after 3.0.0.

peterphilipp commented 7 months ago

Does anyone knows about simple (and free) KNX debug tool that would display KNXnet/IP datagram ? Something like MQTT explorer but for KNX ? Doesn't need to be GUI, command line is fine too May be Wireshark has a KNX décoder ?

Hi, I use this nodered node to debug my KNX Bus: https://flows.nodered.org/node/node-red-contrib-knx-ultimate

peterphilipp commented 7 months ago

Hi, thank you for your effort! Please tell me if I can help with the tests...

barbudor commented 7 months ago

Hi @peterphilipp Could you please provide me with a screen shot of your KNX configuration page ? Just to be sure as I'm not a KNX user Thanks

barbudor commented 7 months ago

First quick test For now my PC is unable to get anything on the multicast group wether using Wireshark or nodered... May be a wifi configuration problem regarding multicast

If you can try in the meantime, here is a zip witha tasmota-knx.bin.gz inside tasmota-knx(test1).zip I'll try again tomorrow evening

peterphilipp commented 7 months ago

Hi, here are my nodered KNX configuration node settings:

grafik

I could not the version, the upgrade by file upload failed with this error message:

grafik

Noschvie commented 7 months ago

@peterphilipp you have to upload the minimal version before. barbudor requested the Tasmota KNX configuration page.

Using these settings for tests: grafik

peterphilipp commented 7 months ago

Ah, sorry, this is the Tasmota Configuration Panel: grafik

The Upload still does not work even after downgrading to the minimal version. I am still getting the same error message.

barbudor commented 7 months ago

$00000023 => 35 Wh

I setup 2 plugs, the first one connected to an TP Link Archer router, and the 2nd one to an Xiaomi router. Boht used as simple AP, not routers. I can't receive the one connected to the TP-Link but I can receive the one connected to the Xiami router I haven't been able to find a setting related to IGMP in neither of those but it seems to be natively supported in the Xiaomi router

I haven't been able to get the NodeRED KNX Ultimate to work because my NodeRED is in a docker and apparently multicast to/from docker is not simple

I'm looking forward the results of your tests

Noschvie commented 6 months ago

@barbudor want to test your changes with an ESP32c3 (Athom Plug V3). How did you change the code casting float to uint32?

knx.answer_4byte_float(msg.received_on, Energy->yesterday_sum); Thanks!

barbudor commented 6 months ago

See the changes in the draft PR here https://github.com/arendst/Tasmota/pull/21074/files

barbudor commented 6 months ago

I just made a refactoring around KNX enhancement

peterphilipp commented 6 months ago

Hi, I did some tests, and it looks great:

grafik

From my point of view, the test is a success. That is a great help for a further homeassistant integration! Thanks...

barbudor commented 6 months ago

Waiting for @Noschvie tests on C3 to move the PR from draft to read-for-review

Noschvie commented 6 months ago

@barbudor suggest to use round() for "1000.0 * " like this

round(1000.0 * Energy->yesterday_sum)

using round will provide the same value as over MQTT.

grafik

16:42:46.968 MQT: tele/tasmota_F982B0/SENSOR = {"Time":"2024-03-31T16:42:46","ENERGY":{"TotalStartTime":"2024-03-09T11:00:02","Total":0.152,"Yesterday":0.060,"Today":0.092,"Period":0,"Power":0,"ApparentPower":0,"ReactivePower":0,"Factor":0.00,"Voltage":237,"Current":0.000}}

the value for "Voltage" should be the same as for MQTT, maybe rounded in both cases?

Thanks for the PR!

barbudor commented 6 months ago

In your example, MQTT voltage is rounded because you have chosen VoltRes 0, so round() is not a good solution for user who chose a different VoltRes For Energy, it's controlled by EnergyRes but based on kWh, not Wh

Let me think if I can find something smart there without increasing the code base For MQTT this is done in ext_vsnprintf_P() while building the JSON string Note that applying rounding on floats may not provide the same result neither

barbudor commented 6 months ago

So for Energy, hte highest value is EnergyRes 3 which give the Wh resolution I can do round() but if anyone use EnergyRes 0, 1 or 2, then value won't match Still it make sense

For Volt/Amp/Watt/Temp/Hum, there no easy way so I'll leave as it for now

Noschvie commented 6 months ago

... understand and ok for me, thanks again!

pbrinette commented 5 months ago

Since this commit, the whole KNX configuration is lost after reboot when configuring a DHT11 sensor as KNX Source.

If I don't configure the sensor as a KNX object, the configuration is persistant after the reboot.

I.E : Configure 2 GA

13:06:06.021 HTP: Configure KNX
13:06:06.023 KNX: Enabled: 1, Communication Enhancement: 0
13:06:06.024 KNX: Physical Address: 0.2.178
13:06:06.026 KNX: GA: 2
13:06:06.027 KNX: GA #1: KNX TX 1 to 0/6/110
13:06:06.029 KNX: GA #2: KNX TX 2 to 0/6/111
13:06:06.030 KNX: CB: 0
13:06:06.031 HTP: Configuration

Configuration is still present after reboot

When I configure the DHT11 sensor to send data on a KNX GA, the configuration is lost after the reboot.

Adding 2 GA for Temperature and Humidity sensors

13:10:29.454 HTP: Configure KNX
13:10:29.456 KNX: Enabled: 1, Communication Enhancement: 0
13:10:29.458 KNX: Physical Address: 0.2.178
13:10:29.460 KNX: GA: 4
13:10:29.461 KNX: GA #1: KNX TX 1 to 0/6/110
13:10:29.462 KNX: GA #2: KNX TX 2 to 0/6/111
13:10:29.464 KNX: GA #3: Temperature to 0/6/106
13:10:29.466 KNX: GA #4: Humidity to 0/6/107
13:10:29.467 KNX: CB: 0
13:10:29.468 HTP: Configuration

This configuration works as expected until reboot .

13:12:17.924 KNX: Temperature sent to 0/6/106
13:12:17.930 KNX: Humidity sent to 0/6/107
13:12:17.938 RSL: SENSOR = {"Time":"2024-04-10T13:12:17","DHT11":{"Temperature":1.2,"Humidity":3.0,"DewPoint":-38.0},"TempUnit":"C"}

After reboot, All KNX configuration is lost except the physical address

13:15:21.965 HTP: Configure KNX
13:15:21.968 KNX: Enabled: 1, Communication Enhancement: 0
13:15:21.970 KNX: Physical Address: 0.2.178
13:15:21.972 KNX: GA: 0
13:15:21.973 KNX: CB: 0
13:15:21.975 HTP: Configuration

I've done the same test with the previous code from the master branch (v13.4.0) and the configuration is persistant.

barbudor commented 5 months ago

Hello @pbrinette Could you please kindly open a new Issue ? Please provide logs at reboot with at least loglevel 3 or 4 I'm not sure I have a DHT11 to test, I will see what I can do

arendst / Tasmota