Koenkk / zigbee-herdsman-converters

Collection of device converters to be used with zigbee-herdsman
MIT License
882 stars 2.94k forks source link

Help to get better integration for Viessmann Valve #1242

Closed ChrisChoke closed 3 years ago

ChrisChoke commented 4 years ago

Hey all,

I integrated the new Viessmann Thermostat Valve. I have some questions for the received data.

The piHeatingDemand data should be between 0-255. and z2m use some math to get percent value. Well the Thermostat seems to just send a payload between 0-5. have you any ideas how to debug that for more details?! i have a pcap with wireshark if its helpful. The Payload what z2m published to mqtt is always "received data" / 255 *100. if i received the data with 1. the mqtt payload is = 0.

Then i have attributes names 16384 with payload between 0 -3. it seems to be a manufactur specific attribute. the attributID is 0x4000. that i have sniffed with wireshark. But i need some help to come deeper in debuging and understanding the zigbee data what i can see in wireshark :-) Is there a possibilty to get to know what this attribute means?

how can i get the Firmware URL for OTA updates? can it be sniffed? or should i ask the manufacturer?

here the database.db entry:

{"id":4,"type":"EndDevice","ieeeAddr":"0x90fd9ffffe4a620e","nwkAddr":15806,"manufId":4641,"manufName":"Viessmann","powerSource":"Battery","modelId":"7637434","epList":[1],"endpoints":{"1":{"profId":260,"epId":1,"devId":769,"inClusterList":[0,1,3,10,32,513,516,2821],"outClusterList":[0,25],"clusters":{"genBasic":{"attributes":{"modelId":"7637434","manufacturerName":"Viessmann","powerSource":3,"zclVersion":3,"appVersion":0,"stackVersion":8,"hwVersion":69,"dateCode":"20190705","swBuildId":"01.12.0008 02.00"}},"hvacThermostat":{"attributes":{"16384":1,"16391":0,"16392":0,"localTemp":2216,"occupiedHeatingSetpoint":2100,"pIHeatingDemand":2,"systemMode":4,"ctrlSeqeOfOper":2}},"genPowerCfg":{"attributes":{"batteryPercentageRemaining":158}}},"binds":[{"cluster":0,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1},{"cluster":1,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1},{"cluster":3,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1},{"cluster":10,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1},{"cluster":32,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1},{"cluster":513,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1},{"cluster":516,"type":"endpoint","deviceIeeeAddress":"0x00124b0018e1ea90","endpointID":1}]}},"appVersion":0,"stackVersion":8,"hwVersion":69,"dateCode":"20190705","swBuildId":"01.12.0008 02.00","zclVersion":3,"interviewCompleted":true,"meta":{"configured":1},"lastSeen":1588708734905}

i would be happy if we could discuss this points and when i could learn more about zigbee and get deeper knowing :-)

i could provide a pcap from my sniffing. But i just sniffed the zigbee2mqtt network. not with the original bridge.

best regards Chris

Koenkk commented 4 years ago

Well the Thermostat seems to just send a payload between 0-5. have you any ideas how to debug that for more details?!

Try playing a bit with https://github.com/Koenkk/zigbee-herdsman-converters/blob/67b6692aba5c436ae844b3b5b1626caa27828e3b/converters/fromZigbee.js#L2577 , maybe e.g. result.pi_heating_demand = msg.data['pIHeatingDemand']; gives the correct values.

Is there a possibilty to get to know what this attribute means?

Can be hard, easiest way to figure this out is asking the manufacturer.

how can i get the Firmware URL for OTA updates? can it be sniffed?

First you have to make sure they provide firmware updates. If yes:

ChrisChoke commented 4 years ago

maybe e.g. result.pi_heating_demand = msg.data['pIHeatingDemand']; gives the correct values.

i tried. and yeah the mqtt message will send the payload what i received.

debug 2020-05-18 20:19:45: Received Zigbee message from 'vicarethermostat', type 'attributeReport', cluster 'hvacThermostat', data '{"pIHeatingDemand":1}' from endpoint 1 with groupID 0
info  2020-05-18 20:19:45: MQTT publish: topic 'zigbee2mqtt/vicarethermostat/pi_heating_demand', payload '1'

and before with precisionround

debug 2020-04-30 05:14:05: Received Zigbee message from 'vicarethermostat', type 'attributeReport', cluster 'hvacThermostat', data '{"pIHeatingDemand":4}' from endpoint 1 with groupID 0
info  2020-04-30 05:14:05: MQTT publish: topic 'zigbee2mqtt/vicarethermostat/pi_heating_demand', payload '2'

Do you think that the payload is already in percent how its defined in the ZCL? https://zigbeealliance.org/wp-content/uploads/2019/12/07-5123-06-zigbee-cluster-library-specification.pdf Page 372. May i ask why we use precisionround in stable release?! the ZCL defined possible values between 0-100 (0x00 – 0x64). How it can be that seemingly some manufacturer can send values between 0-255? :-) Well i never saw values higher than 5 at my thermostat. So i am not sure that the manufactur use the ZCL defined values...is such a szenario possible?

Can be hard, easiest way to figure this out is asking the manufacturer.

Do you have any experience that the manufacturer are disposed to get those infomarations free? i can think they would bind their customers at their own bridge :-)

sjorge commented 4 years ago

It’s sad that for TRV nearly nobody seems to be following the ZCL.

Having tried a few and liking none, I have the most experiences it’s the eurotronic ones which send 0-255 for the %, Also with the SE iTRV which sends a normal percentage 0-100...

I think manufacturer sadly interpret the ZCL different, one use the range other just the datatype... and tuya implement a custom protocol ontop of zigbee :(

As you are already hinted at most don’t seem very helpful when you ask questions, SE ignored me. Eurotronic kindly told me to take a hike.

I suppose you could probe the range by setting it to a super low value so the TRV closes the valve and to the max so it will fully open it.

If it indeed only outputs 0-5 it might just indicate a sort of 10% increase or something, which would suck.

ChrisChoke commented 4 years ago

well i am not a programmer. but have highly interest on learning a bit about it. i am just a electrician. and for me the ZCL is really clear. i can read there is "possible value from 0-100" they just write it down in hex. so even me understand what the documentation means:-) hehe.

i send an email and hope getting positiv answer. Viessmann buyed Wibutler for few years ago. and for that reason they integrated their own products in wibutler. wibutler is a hub/bridge that can communicate with zigbee,z-wave,enocean, bluetooth and tcp. And I mean Viessmann provide a API to get the data from your heat generator, if you have a login to an developer portal. i read something about it in their forum. its let me hope that they are a bit open for that. if i a manufactur i would support the open source comminty. its a possibility to sell more of my devices as if i try to bind people at my proprietary bridge.

i tried your suggestion and i got only values between 0-5. I hope getting an answer of my mail :-) it would be sad cause i like that plain design and i would like to buy 6 additional TRVs.

i keep you up to date!!

edit: does someone have a login to the zigbeealliance?! they provide a database with all or the most information about certified zigbee devices and their features.

sjorge commented 4 years ago

Yeah they look really interesting, so far I tried a few and none I really liked. I for one on order but it will take 8 weeks :|

~ sjorge

On 19 May 2020, at 21:56, ChrisChoke notifications@github.com wrote:

 well i am not a programmer. but have highly interest on learning a bit about it. i am just a electrician. and for me the ZCL is really clear. i can read there is "possible value from 0-100" they just write it down in hex. so even me understand what the documentation means:-) hehe.

i send an email and hope getting positiv answer. Viessmann buyed Wibutler for few years ago. and for that reason they integrated their own products in wibutler. wibutler is a hub/bridge that can communicate with zigbee,z-wave,enocean, bluetooth and tcp. And I mean Viessmann provide a API to get the data from your heat generator, if you have a login to an developer portal. i read something about it in their forum. its let me hope that they are a bit open for that. if i a manufactur i would support the open source comminty. its a possibility to sell more of my devices as if i try to bind people at my proprietary bridge.

i tried your suggestion and i got only values between 0-5. I hope getting an answer of my mail :-) it would be sad cause i like that plain design and i would like to buy 6 additional TRVs.

i keep you up to date!!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

ChrisChoke commented 4 years ago

short update. I received an answer from a account manager from Viessman who means that they dont give any information. short answer. just 5 words... I try another request to a manager which is a digital innovation manager. he is active in their community for an API. so i hope he could be help a bit more.

waiting for answer and keep you up to date. the last chance i will buy a manufacture bridge and try to reengineering if the answer will not as expected.

Chris

sjorge commented 4 years ago

Mine arrived today, no gateway as the TRV alone was already 150 EUR with shipping! Well shipping was like 75 so if I get more it will become cheaper. Won't have time to play with it this weekend I fear as it looks like it will be busy.

But the device does feel 'nice', at least compare to the eutronic and tuya ones.

ChrisChoke commented 4 years ago

What device you ordered? The Viessmann trv?

sjorge commented 4 years ago

Yes, as I am still looking for a TRV to use in my new appartment (still being build) but none of the options so far have had me satisfied.

eurotronic: best so far, but feels cheap and has some odd qwerkt with boost/openwindow tuya: feels cheap, custom tuya stuff, no piHeatingDemand schnieder electric: feels solid, very flaky, almost never responds ecosy: not able to find, seems discontinued

So fingers crossed, as far as I could tell it's mostly sticking to the spec... aside from piHeatingDemand but I'm ok with > 0 for open and 0 for closed :D

sjorge commented 4 years ago

Good news, I already found a few things to improve!

I noticed the HA snippet in the docs mentions system_mode but we currently cant set or get it. Adding 'tz.thermostat_system_mode' allowed me to query it. I will also add pairing information to the docs. Will poke it some more.

sjorge commented 4 years ago

@Koenkk after adding the tz.thermostat_system_mode to devices.js I can successfully query the device and it returns 'heat'

But I cannot set the system_mode I always get the error, I tried off and heat.

zigbee2mqtt:error 2020-06-11 13:38:48: Publish 'set' 'system_mode' to 'bedroom/radiator' failed: 'Error: Write 0x90fd9ffffe4a612f/1 hvacThermostat({"systemMode":"0"}, {"timeout":10000,"disableResponse":false,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null}) failed (Error: Status 'INVALID_VALUE')'

I tried with payload of 0, off, 4 and heat... one of these should have been OK right? https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/converters/common.js#L11

I don't see the coordinator sending the message at all, so I don't think the TRV is rejecting it.

Nope, the TRV is rejecting it. 0/off fails, but 4/heat does work! I had use 2 and head because I can't spell and misread the enum. I'm pretty sure the HA integration for this device won't work the as system_mode can't be changed, nor does it publish it via att_report. (It can be read though)

sjorge commented 4 years ago

I do see it send a few unknown fields:

zigbee2mqtt:debug 2020-06-11 13:10:01: Received Zigbee message from '0x90fd9ffffe4a612f', type 'attributeReport', cluster 'genIdentify', data '{"16384":0}' from endpoint 1 with groupID 0
zigbee2mqtt:debug 2020-06-11 13:10:01: No converter available for 'ZK03840' with cluster 'genIdentify' and type 'attributeReport' and data '{"16384":0}'
zigbee2mqtt:debug 2020-06-11 15:07:58: Received Zigbee message from 'bedroom/radiator', type 'attributeReport', cluster 'hvacThermostat', data '{"16384":2}' from endpoint 1 with groupID 0
zigbee2mqtt:debug 2020-06-11 15:08:58: Received Zigbee message from 'bedroom/radiator', type 'attributeReport', cluster 'hvacThermostat', data '{"16384":1}' from endpoint 1 with groupID 0

And I can confirm pi_heating_demand is 0-5 :s

ChrisChoke commented 4 years ago

The last HVAC data 16384 means zigbee attribute id 0x4000. It's manufacturer specific. I guess it is for open window detection. Look on the trv display. You should see "00". That means the trv detects a open window and stop heating till the temperature in the room grow up or 30 minutes later.

I think I am at 7pm on computer. I send further Infos I have made. toZigbee control sequenz....works as well. You can query the global mode.

ChrisChoke commented 4 years ago

here my device.js config for this device. all that toZigbee converters works with a response.

control_sequenz_of_operation get "heating only" occupied_cooling_setpoint get "0" system_mode get "heat" if you request with a "get" request via mqtt. do not wonder at the fromZigbee. i create a new converter for playing a bit. the other toZigbee converters seems it not defined.

{
        zigbeeModel: ['7637434'],
        model: 'ZK03840',
        vendor: 'Viessmann',
        description: 'ViCare radiator thermostat valve',
        supports: 'thermostat',
        fromZigbee: [fz.viessmann_att_report, fz.battery],
        toZigbee: [tz.thermostat_occupied_heating_setpoint,
                   tz.thermostat_occupied_cooling_setpoint,
                   tz.thermostat_control_sequence_of_operation,
                   tz.thermostat_system_mode,
                   tz.viessmann_open_window
                ],
        meta: {configureKey: 1},
        configure: async (device, coordinatorEndpoint) => {
            const endpoint = device.getEndpoint(1);
            await bind(endpoint, coordinatorEndpoint, [
                'genBasic', 'genPowerCfg', 'genIdentify', 'genTime', 'genPollCtrl', 'hvacThermostat',
                'hvacUserInterfaceCfg',
            ]);
            await configureReporting.thermostatTemperature(endpoint);
            await configureReporting.thermostatOccupiedHeatingSetpoint(endpoint);
            await configureReporting.thermostatPIHeatingDemand(endpoint);
        },
    },

here a picture from a wireshark sniff for the 16384 attribut which you saw:

Bildschirmfoto von 2020-06-11 20-08-34

and here a example from response for remote_sensing. it is not defined. you can see it in under "status record"

Bildschirmfoto von 2020-06-11 20-15-34

i tried to configure a toZigbee converter for the attribute 16384 to request the status but i get an "unsupported attribute" response in z2m log.

here the converter:

viessmann_open_window: {
        key: ['open_window'],
        convertSet: async (entity, key, value, meta) => {
            const payload = {0x4000: {value, type: 0x30}};
            await entity.write('hvacThermostat', payload);
        },
        convertGet: async (entity, key, meta) => {
            await entity.read('hvacThermostat', [0x4000]);
        },
    },

i try to play a bit. maybe i can get it work. i will see.

chris

sjorge commented 4 years ago

I think I can help with the but about unsupported attribute, let me dig up what I think is causing it later.

Yeah I noticed system_mode does work, but it only has one valid value 'heat'. Not sure how useful it is to expose it though.

I wonder if occupied_cooling_setpoint is a left over bug somehow, given that it only takes 'heat' as operation mode, having a cooling_setpoint makes... no sense?

For window open I assume in viessmann_thermostat_att_report you also parse the attribute?

sjorge commented 4 years ago

So for the attribute error you are getting, I think the manufacturer cluster needs to be added to herdsman. e.g. https://github.com/Koenkk/zigbee-herdsman/blob/e3ac871e8f43b2ebb2287612bad2268fbf3abca8/src/zcl/definition/cluster.ts#L3705

ChrisChoke commented 4 years ago

Bildschirmfoto von 2020-06-11 21-13-53

yeah it seem that the system_mode can just have this static value. tried auto and emergency heating to write. it sending the correct Uint8 data, but the device responding invalid value.

i would try adding the unsupported attribute to cluster. ts. but how :-D can't found the file in my z2m environment.

For window open I assume in viessmann_thermostat_att_report you also parse the attribute?

yeah.

viessmann_att_report: {
        cluster: 'hvacThermostat',
        type: ['attributeReport', 'readResponse'],
        convert: (model, msg, publish, options, meta) => {
            const result = {};
            if (typeof msg.data['localTemp'] == 'number') {
                result.local_temperature = precisionRound(msg.data['localTemp'], 2) / 100;
            }
            if (typeof msg.data['occupiedHeatingSetpoint'] == 'number') {
                const ohs = precisionRound(msg.data['occupiedHeatingSetpoint'], 2) / 100;
                if (ohs < -250) {
                    // Stelpro will return -325.65 when set to off
                    result.occupied_heating_setpoint = 0;
                } else {
                    result.occupied_heating_setpoint = ohs;
                }
            }
            if (typeof msg.data['occupiedCoolingSetpoint'] == 'number') {
                result.occupied_cooling_setpoint =
                    precisionRound(msg.data['occupiedCoolingSetpoint'], 2) / 100;
            }
            const ctrl = msg.data['ctrlSeqeOfOper'];
            if (typeof ctrl == 'number' && common.thermostatControlSequenceOfOperations.hasOwnProperty(ctrl)) {
                result.control_sequence_of_operation = common.thermostatControlSequenceOfOperations[ctrl];
            }
            const smode = msg.data['systemMode'];
            if (typeof smode == 'number' && common.thermostatSystemModes.hasOwnProperty(smode)) {
                result.system_mode = common.thermostatSystemModes[smode];
            }
            const owindow = msg.data[0x4000];
            if (typeof owindow == 'number' && common.manufacturerOpenWindow.hasOwnProperty(owindow)) {
                result.open_window = common.manufacturerOpenWindow[owindow];
            }
            if (typeof msg.data['pIHeatingDemand'] == 'number') {
                result.pi_heating_demand = msg.data['pIHeatingDemand'];
            }
            return result;
        },
    },

and i add a mapping in common.js

on discord there is a user names Zeddy who try integrating a danfoss abby trv. his documentation says:

0x4000 -> eTRV Window Open Detection -> enum8 -> R

and he show me the value mapping from this device. so maybe its the same like here.

sjorge commented 4 years ago

You should be able to do something like, this reuses the code from thermostat_att_report and cuts back on duplication.

    viessmann_thermostat_att_report: {
        cluster: 'hvacThermostat',
        type: ['attributeReport', 'readResponse'],
        convert: (model, msg, publish, options, meta) => {
            const result = converters.thermostat_att_report.convert(model, msg, publish, options, meta);

            // ViessMann TRVs seems to report pIHeatingDemand between 0 and 5 already
            if (typeof msg.data['pIHeatingDemand'] == 'number') {
                result.pi_heating_demand = precisionRound(msg.data['pIHeatingDemand'], 0);
            }

            // window open using manufacturer specific attribute
            if (typeof msg.data['16384'] == 'number' && common.manufacturerOpenWindow.hasOwnProperty(msg.data[16384] )) {
                result.window_open = common.manufacturerOpenWindow[msg.data['16384'] ];
            }
            return result;
        },
    },

Note: I picked 'window_open' as it is already use for other devices. Edit: I think the number needs to be quotes, not 100% sure, not near the device to test

ChrisChoke commented 4 years ago

i have it. had to configure the manuFacturerCode in toZigbee.js

viessmann_open_window: {
        key: ['open_window'],
        convertSet: async (entity, key, value, meta) => {
            const payload = {0x4000: {value, type: 0x30}};
            await entity.write('hvacThermostat', payload, options.viessmann);
        },
        convertGet: async (entity, key, meta) => {
            await entity.read('hvacThermostat', [0x4000], options.viessmann);
        },
    },

and the manuFacturerCode definition in the same file:

const options = {
    xiaomi: {
        manufacturerCode: 0x115F,
        disableDefaultResponse: true,
    },
    osram: {
        manufacturerCode: 0x110c,
    },
    eurotronic: {
        manufacturerCode: 4151,
    },
    hue: {
        manufacturerCode: 4107,
    },
    sinope: {
        manufacturerCode: 0x119C,
    },
    ubisys: {
        manufacturerCode: 0x10f2,
    },
    tint: {
        manufacturerCode: 0x121b,
    },
    legrand: {
        manufacturerCode: 0x1021,
        disableDefaultResponse: true,
    },
    viessmann: {
        manufacturerCode: 0x1221,
    },
};

Bildschirmfoto von 2020-06-11 21-33-41

so its fine at the moment. i can query this attribute.

ChrisChoke commented 4 years ago

You should be able to do something like, this reuses the code from thermostat_att_report and cuts back on duplication.

    viessmann_thermostat_att_report: {
        cluster: 'hvacThermostat',
        type: ['attributeReport', 'readResponse'],
        convert: (model, msg, publish, options, meta) => {
            const result = converters.thermostat_att_report.convert(model, msg, publish, options, meta);

            // ViessMann TRVs seems to report pIHeatingDemand between 0 and 5 already
            if (typeof msg.data['pIHeatingDemand'] == 'number') {
                result.pi_heating_demand = precisionRound(msg.data['pIHeatingDemand'], 0);
            }

            // window open using manufacturer specific attribute
            if (typeof msg.data['16384'] == 'number' && common.manufacturerOpenWindow.hasOwnProperty(msg.data[16384] )) {
                result.window_open = common.manufacturerOpenWindow[msg.data['16384'] ];
            }
            return result;
        },
    },

Note: I picked 'window_open' as it is already use for other devices. Edit: I think the number needs to be quotes, not 100% sure, not near the device to test

yep it works as you descriped. just copy and paste and it works. we just think about what the best between msg.data['16384'] and msg.data[0x4000]

dont know. how i had code it, i just saw the same way in the actual codebase. is there a standard?

thank you for your tip for reusing existing code. that i in my sense too.

precisionRound(msg.data['pIHeatingDemand'], 0); and (msg.data['pIHeatingDemand']; do the same result. its recommend to use the precisionRound though?

sjorge commented 4 years ago

I just left the percisionRound in there, although it can probably go I think.

I used what was show in the debug log and it showed "16384" as a string, so it's easier to make the connection. Seeing "16384" in the debug log but 0x4000 in the converter does not immediately make it obvious it is the same thing.

sjorge commented 4 years ago

Are you actually getting anything form ctrlSeqeOfOper ?

If so you could maybe try and configure the reporting for it.

The follow can be added to const configureReporting = { in devices.js

    thermostatControlSequanceOfOperation: async (endpoint, min=0, max=repInterval.HOUR, change=1) => {
        const payload = [{
            attribute: 'ctrlSeqeOfOper',
            minimumReportInterval: min,
            maximumReportInterval: max,
            reportableChange: change,
        }];
        await endpoint.configureReporting('hvacThermostat', payload);
    },

Then call it and bumb the configureKey in the device block

await configureReporting.thermostatControlSequanceOfOperation(endpoint, 0, repInterval.MINUTES_15, 1);

I get no response when trying to read or set this attribute manual either though.

ChrisChoke commented 4 years ago

I used what was show in the debug log and it showed "16384" as a string, so it's easier to make the connection. Seeing "16384" in the debug log but 0x4000 in the converter does not immediately make it obvious it is the same thing.

okay that make sense.

i've testet now all attributes again. My results:

getting a "read attributes response" with "unsupported attribute":

tz.thermostat_occupancy
tz.thermostat_running_mode
tz.thermostat_running_state
tz.thermostat_remote_sensing
tz.thermostat_temperature_setpoint_hold
tz.thermostat_temperature_setpoint_hold_duration
tz.thermostat_local_temperature_calibration
tz.thermostat_unoccupied_cooling_setpoint
tz.thermostat_unoccupied_heating_setpoint

getting just a "default response" with "unsupported cluster command":

tz.thermostat_weekly_schedule
tz.thermostat_relay_status_log

the working attributes are:

tz.thermostat_local_temperature,
tz.thermostat_occupied_heating_setpoint,
tz.thermostat_occupied_cooling_setpoint,
tz.thermostat_control_sequence_of_operation,
tz.thermostat_system_mode,

this device have 3 manufacturer specific attributes which i know:

16384/ 0x4000            enum8 ( 0x30 ), read only
16391/ 0x4007            boolean ( 0x10 ), read only
16392/ 0x4008            Uint16 ( 0x21 ), read only

tz.thermostat_occupied_cooling_setpoint you can write a value but z2m dont published it via mqtt like at thermostat_occupied_heating_setpoint. but you can read it via "get" command. in wireshark you can see that the writing is suceed. dont know where is the sense.

sjorge commented 4 years ago

For cooling_setpoint you also need to add reporting config, I show you an example for control_sequence_of_operation a post ago. Should be the same for cooling_setpoint but I think there might already be an existing one so check for that first.

I repaired (well removed the batteries, I have not found how to successfully reset the device) and also observe 1x reporting of

zigbee2mqtt:debug 2020-06-12 20:52:52: Received Zigbee message from 'bedroom/radiator', type 'attributeReport', cluster 'hvacThermostat', data '{"16384":0,"16391":0,"16392":0}' from endpoint 1 with groupID 0
sjorge commented 4 years ago

Good news though, I just got a attReport for control_sequence_of_operation! It happened ~ the hour mark as configure with the example above. Let me try with system_mode too.

We can probably do the same for the window open and other 2 values, the main question is... do we want reporting for system_mode and control_sequence_of_operation... as they do not seem to change at all? they are 'heat' and 'heating only' respectively.

ChrisChoke commented 4 years ago

well i am in waiting time for the report at the moment :-) have add the systemMode and coolingSetpoint which was already configured in the const configureReporting, too. i will see if i get reporting. for the window_open we can do it. but dont know at the moment if my mapping of the value interpreting is right. i think i will ask viessman directly with this specific attributes. the inovation manager seems to be open.

edit: @sjorge here for you: https://webapps.viessmann.com/vibooks/api-internal/file/resources/technical_documents/DE/de/VBA/5802069VBA00002_1.PDF?#pagemode=bookmarks&zoom=page-fit&view=Fit

sjorge commented 4 years ago

Would be nice to know what the piHeatingDemand mapps to as i only seem to get: 0, 1, 2, 5

sjorge commented 4 years ago

Handing link, sadly my german is bad. I wonder if I can guess the url for english as there is DE/de in the url.

ChrisChoke commented 4 years ago

haha :-D yeah i am sorry. it seems there is no documentation in another language. neither belgium - netherlands and belgium - francais. you only have the quick entry document which was in the paket from the device, right?!

for factory reset you have to push the pairing button and put in the battery. dont leave the button. all display lights will glow/flash for 2 seconds. after that you see "- -" in the display. the mounting mode is activ and the device can re-pair.

sjorge commented 4 years ago

Yeah it just, doesn't even have 'text' except the warnings the instructions are just... pictures. Meanwhile, I was suppose to go to bed but I am crawling there tech site endpoint:

https://webapps.viessmann.com/vibooks/api-internal/file/resources/technical_documents/GB/en/VKE/4754331VKE00001_1.PDF http://static.viessmann.com/resources/technical_documents/GB/en/VMA/5800251VMA00001_1.PDF (got this one, or the next not sure) http://static.viessmann.com/resources/technical_documents/GB/en/VMA/5800251VMA00002_1.PDF (got this one, or the previous not sure)

But nothing as extensive as the german one. Looks like VBA dir holds 'installer' manuals, the german one has a lot more files, but given ViessMann is a german company that makes sense.

sjorge commented 4 years ago

Some observations, reporting seems to work fine for system_mode and control_sequence_of_operation.

It is setup correctly for occupied_cooling_setpoint, but that value seems to have no effect on the operation?

I did not have luck setting reporting for the 3 manifacurar values, but that is because I did not modify herdsmen yet to recognize them.

And yeah I am also on discord, but I only check there once every few days. I'm more active here.

ChrisChoke commented 4 years ago

i can confirm that the reporting of systemMode and control_sequence_of_operating is working. the occupied_cooling_setpoint also. but i dont know what it do. i cant see any effects if i set the cooling setpoint. but why there isnt a "change"-value configured in the systemMode at const configureReporting

but i figure out the tz.thermostat_keypad_lockout works, too. possible values are "1" for lockout and "0" for no lockout! the values "2" - "5" run in "invalid value". the effect is you cant change the temperature at the device with the adjusting wheel.

setting the reporting for manufacturer attributes i will try in the course of the week. i think we have done a lot this weekend and i have to say thank you @sjorge . i've learned a lot with your help. you pointing me a lot information to find their way in the code.

edit: if i set the keypad_lockout i get just the integer in the payload of the mqtt message even though in common.js there is a mapping to a string. any idea?! i will observe it further.

Received Zigbee message from 'vicarethermostat', type 'attributeReport', cluster 'hvacUserInterfaceCfg', data '{"keypadLockout":0}' from endpoint 1 with groupID 0

MQTT publish: topic 'zigbee2mqtt/vicarethermostat/keypad_lockout', payload '0'

sjorge commented 4 years ago

Oh nice that i also support keypad lockout.

I think we should probably slowly start adding this, but only things that make sense.

Not sure on the occupied_cooling_setpoint, if it doesn't work. We probably should not add it until we figure it out.

There is actually already a function for system_mode reporting! It's control_sequence_of_operation that is missing, but as mentioned above if the value is never going to change, no use in reporting on it.

I also noticed the device doe OTA queries! if we somehow can get the encrypte firmware images we can easily add them and have the valve update themselves!

I also tried to dig into pi_heating_demand, but not much luck. Raw values are 0-5 or me, but it's nearly always 0 -> 5 -> 1 -> 0, where the 1 -> 0 transition takes ages.

The odd thing is, it never opened the valve during the change 🤷‍♂️ So it's not like we can map this to something resembling a % of valve open as we should expect. Some valve either return 0-100 or 0-255 so it depends on how the manufacturer interprest the ZCL, but 0-5 is just plain weird and probably wrong.

Maybe they ship with a bad firmware that the gateway immediately updates? (Also if you sniff the traffic and we figure out the network key, we can probably extract the firmware from the gateway if it does update them)

ChrisChoke commented 4 years ago

Well I agree. I will prepare a commit and pull request. I will add all working attriubutes to the toZigbee section. but just configure a reporting for:

await configureReporting.thermostatTemperature(endpoint);
await configureReporting.thermostatOccupiedHeatingSetpoint(endpoint);
await configureReporting.thermostatOccupiedCoolingSetpoint(endpoint);
await configureReporting.thermostatPIHeatingDemand(endpoint);
await configureReporting.thermostatKeypadLockMode(endpoint);

although i think the occupiedcooling setpoint makes no sense, too. if there is no effect for the device :-)

Ive send an email to the manufacturer to get information about the manufacturer specific attributes. i would wait for an answer a few days. i propose we wait to friday. If i dont get more information i will push the new config without the window_open attribute. @sjorge would you want to review before i send pull request to Koenks master?

question about keypadlockout: is it wanted that here does not send the mapping strings which is defined in common.js for this attribute? @Koenkk maybe you know more about it.

https://github.com/Koenkk/zigbee-herdsman-converters/blob/7b769ef58f3667fec6691c14b071932c92037eb5/converters/fromZigbee.js#L2744-L2746

ChrisChoke commented 4 years ago

Hey hey,

i updated the code for the trv. @sjorge can you take a look? at the moment the code cant automaticly merged. for my shame, i dont know how to get the actual codebase without throw away my changes to make a pull request which can automaticly merged :-(

https://github.com/ChrisChoke/zigbee-herdsman-converters/commit/e6a2e679339a0dd3b03b574a3f95d2d325b5e3ce

sjorge commented 4 years ago

Bit busy at the moment, but I will try to look at it on Wednesday, poke me if I forget.

How good are you with git? You can try something like

# add koenkk's repo as another remote called 'upstream'
git remote add upstream https://github.com/Koenkk/zigbee-herdsman-converters
# create a copy of your branch 
git checkout -b dev-vicareTrv-rebase
# fetch latest version of 'dev' from koenkk's repo
git fetch upstream/dev
# attempt to rebase 
git rebase upstream/dev

We make a copy of your branch so even if you mess up you can just go back with git checkout dev-vicareTrv

ChrisChoke commented 4 years ago

okay i get it i think. thanks for your help. opened a pull request #1341 its just for the zigbee attributes where we get working. i ordered the original bridge for my heating system. so i will check if we can get the OTA url. and i can check if there is an update in queue.

sjorge commented 4 years ago

Been looking in pIheating demand and all I am getting is 0 or 1 for the last 2 weeks, mostly just 0. It’s been warm so it never had to turn on though. Very annoying.

Did your original bridge arrive?

ChrisChoke commented 4 years ago

Hey sjorge..I am sorry for late response.

The weather was fine and so I have to do my buildingsite in the garden :-)

Yeah the bridge arrived and i did try today to sniff the Network. I think I have found the right zigbee channel. It's 25. But I never saw a device announcement. So I don't have the zigbee transport key. And all message a encrypted. Any tips?

ChrisChoke commented 4 years ago

okay during the pairing process you must scan the QR-Code from the TRV device. the qr include the "eui-64" and the "ic (installation code)". both you will find left from the qr code on the device.

i've attach a pcap from scanning. look for "beacon request" line 2232 and 12968 following. thats the two moments where i haved pair the trv with the vendor bridge. but no "device announcement or transport key" message. the eui-64 from my device is "90:fd:9f:ff:fe:4a:62:0e" look at the "source" column to find some messages.

at the moment i dont know how to figure out the network key if its not showing during the pair process. maybe @Koenkk can help out a bit :-)

will stay on the ball but be grateful for every tips and hints :-)

viessman bridge.zip

ChrisChoke commented 4 years ago

hey guys,

well i think i need some help. i tried to get some information about how i can decrypt the zigbee messages. what i have read about is: The installation code is transformed into a link key by use on an AES MMO hash algorithm. so here can be a possibility to calculate the link key. but how?! :-) And this key seems to be just a temporarily key. it is just use to get the right link key over an encrypted connection. but i if get this temporarily link key, maybe i can see the right link key. does anyone know about that? its a zigbee 3.0 compliant method. need any hinds :-)

the second thing. i have some dns adress from the original vendor bridge. here they are:

zeroseriesupdate.viessmann-platform.io CNAME: vegzsuservice.northeurope.cloudapp.azure.com mgmt.viessmann-platform.io dc1-iothub.azure-devices.net CNAME: ihsu-prod-db-002.cloudapp.net

the bridge requested an update. but it was already installed. the full request was: http://zeroseriesupdate.viessmann-platform.io/firmware/hb2/7637415_Vitoconnect-100_OTOL2.version

so i would be happy if anyone can help. @Koenkk @sjorge

Chris

sjorge commented 4 years ago

bridge URL seems to give 4xx error, so it probably set some cookies or headers. Not sure on how to get the link key :(

https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html seems to have some hints on how to get the keys from the deviceAnnouncement, so maybe try that?

ChrisChoke commented 4 years ago

yeah i have tried the action from the zigbee2mqtt docu, but it does not help in this case. the install code calculate with a mmo (Matyas-Meyer-Oseas) hash function a preshared link key. with this link key i get the real link key from the trusted center. so i need a tool to calculate the preshared key. i have found some function in the zigpy project for that (https://github.com/zigpy/zigpy/blob/14579e5c09082d1bf2c433a344ecefb0673720c4/zigpy/util.py#L208) but dont know ho i can use it. i found a further hint in a master thesis (http://csus-dspace.calstate.edu/bitstream/handle/10211.9/1833/zigbee2.pdf?sequence=2) this guy created a cli tool to get the preconfigured link key. but no chance to get the source code from the link in his thesis.

so its not easy to reach my goal :-)

i am really grateful if anyone can support in the case :-)

ChrisChoke commented 4 years ago

Oh my god!!!! its a nice evening. i decrypted the link key from the install code and i get the new link-key from the trust center.

its like a epiphany :-)

it was quiet easy with this tool and can't thank enough to the owner of the code @jfocht https://github.com/jfocht/install-codes That is what iam search for a long time. its really helpful and works at first try. really nice.

now i see all the zigbee messages and i can say, there are a lot of informations. the TRV has a lot manufacturer specific attributes. i will take detailed look into as soon as possible. at this weekend the time is a bit limited. i saw OTA Request too. if @Koenkk could help which informations are needed i could look for.

Chris

sjorge commented 4 years ago

I'm guessting the brige will download the ota image and push it to the valve, if you capture a full update stream we can extract the image if I am not mistaken.

Also that's good news! Exciting stuff.

github-actions[bot] commented 3 years ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

mweinelt commented 3 years ago

Pretty sure @ChrisChoke is still looking into this.

ChrisChoke commented 3 years ago

okay, hey hey. wish you all a pretty sunday.

i am sitting in the cellar and look for the TRV. so please look in the pcap and in the txt file which is include in the zip. you will find all key you need to decrypt the pcap. i notice some line hinds in the txt file what happend at this points. viessmann_pcap.zip

ChrisChoke commented 3 years ago

i looked at the basic cluster attrubute 0x4000. its SWBuildId. on shipping it was: 01.12.0008 02.00 and now it is: 01.16.0008 02.04

the TRV made an update successful. but i dont know when. it seems it made it in september on my first connecting with the bridge. i hope we can do something with the pcap which presents some ota streams. i really hope someone could look if we can get the firmware out of this. or someone can explain the steps to do it. i didnt find something about file-extracting from wireshark in the world wide web.

i will pair the trv at my z2m system. so maybe i see some new things ^^ maybe the piHeatingDemand works as expected now

Chris

sjorge commented 3 years ago

We can dump the image if it’s in the stream I believe, but if it was update before the capture I don’t think there is much we can do.

~ sjorge

On 1 Nov 2020, at 15:44, ChrisChoke notifications@github.com wrote:

 i looked at the basic cluster attrubute 0x4000. its SWBuildId. on shipping it was: 01.12.0008 02.00 and now it is: 01.16.0008 02.04

the TRV made an update successful. but i dont know when. it seems it made it in september on my first connecting with the bridge. i hope we can do something with the pcap which presents some ota streams. i really hope someone could look if we can get the firmware out of this. or someone can explain the steps to do it. i didnt find something about file-extracting from wireshark in the world wide web.

i will pair the trv at my z2m system. so maybe i see some new things ^^ maybe the piHeatingDemand works as expected now

Chris

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.