DataExtractor

[gismo2004]

@j0ta29 @bertmelis

I have written a simple tool to extract data point information, based on the identifier from VM-database, because I was too lazy to search through the XML files :-)

Now that I have this tool, I thought I could extend it to automatically export the information as optolink yaml.

If you think this is helpful, I would need some input on how the export should look like and what options are availabe.

The attached zip shows my current export as xlsx and json, but since I have the data based on C# classes, it's easy to implement a wrapper for yaml.

2048-V200WO1A.zip

[bertmelis]

Nice. To be honest, I don't have any interest in esphome. I just use my own optolink-mqtt device. Besides, I frequently update to test both VitoWiFi and my MQTT client library.

That said, what VitoWiFi (v3) needs is a name (c-string), an address (2 bytes), length (in bytes) and a conversion type (this also can be "no conversion"). How esphome the yaml converts to C++ code, I don't know.

Is your tool open source?

[gismo2004]

The tool can be made open source of course, the only question to this: how to handle V-Data? I would assume, that I should not ship it with my tool. :-) Maybe I can check if VitoSoft is installed and if yes, use data from there and if not, require it to be placed along with my executable...

For the VitoWiFi export, I think all data is already available in the xlsx export, correct? If not, just tell me which format you want to have it. If you like, I can test drive this based on your device ID? One additional question to VitoWiFi: I have seen some signals that are sharing one message ID, but have different start bits/ length. Is your new V3 able to handle this kind of messages? And what about Arrays?

Regarding the optolink export: my idea was to generate yaml code to be copy and pasted into esphome configuration file. This way it is easy to set up and should also kind of “harmonize” the signal naming. It might be good, to have something to select which messages should be exported to this yaml as I think ~1300 signals are definitely not needed...

[bertmelis]

V3 is more flexible wrt lengths. It just dynamically allocates. There is an option to set message IDs. It's based on this: https://github.com/sarnau/InsideViessmannVitosoft/blob/main/VitosoftCommunication.md

And yes, all required info is available in the xlsx

[j0ta29]

@gismo2004 , below you find a first raw rule set to map the field to a ESPHome configuration. The crucial part is to decide for one of the 6 component types. The rest should be simple mapping. An interesting part will be the mapping of array especially that targeting weekly schedule plans. I see that your device uses 168 bytes. Mine uses 56 bytes. Do you have more information about the interpretation of the 168 byte array?
I'm looking forward to see the results ;-)

component type:
    binary_sensor   -> FCWrite=undefined and FCRead=Virtual_READ and Enumerations with exactly 2 options
    switch          -> FCWrite=Virtual_WRITE and FCRead=Virtual_READ and Enumerations with exactly 2 options
    text_sensor     -> FCWrite=undefined and FCRead=Virtual_READ and ParameterType=String or Enumerations with more than 2 options
    select          -> FCWrite=Virtual_WRITE and FCRead=Virtual_READ and Enumerations with more than 2 options
    sensor          -> FCWrite=undefined and FCRead=Virtual_READ and none of the above component types
    number          -> FCWrite=Virtual_WRITE and FCRead=Virtual_READ and none of the above component types 

Name    
    -> id
Address 
    -> address
PrettyName (column C)   
    -> name
FCWrite 
FCRead
Unit
    -> unit_of_measurement
    -> if °C  -> icon: "mdi:thermometer-water", maybe other mappings
    -> if °C -> device_class: temperature
Conversion
    -> div_ratio 
UpperLimit
    -> min_value (only if component type is 'number')
LowerLimit  
    -> min_value (only if component type is 'number')
Stepping    
    -> step (if not 0)
Priority
ParameterType
   -> bytes
DefaultValue
BitStartPos BitLengh
BlockLength
ByteLength
BytePosition
Enumerations
    -> filters map (if component type is text_sensor)
    -> map (if component type is select)  #

[gismo2004]

@j0ta29 First, a happy new year :-)

Unfortunately it took quite some time to answer as I lost everything I did before (SSD died without backup :-( ), which means, I had to do everything from scratch again! This was quite annoying TBH… however now I am back up with everything and I have attached a first shot of what you suggested above.

Let me know what you think and what to be improved.

2048-V200WO1A.zip

The only information about the array is written in the comment of the xlsx --> I would be happy to check how to decode this arrays, but I don't know how to get a “filled” array to be analyzed.

[j0ta29]

Wow @gismo2004, I am really impressed by the number of sensors etc. you extracted in your example. The extract looks promising. Something, that looks a little bit strange to me: there are entries with div_ratio 0. You know the old rule from school (division by zero)? If the correct value is 1, you can suppress it totally (1 is the default). How comes a div_ratio of 2? For my boiler there is also div_ratio of 3600 in case of datapoints that measure seconds and the sensor has a unit of hours (example "Betriebsstd. ...."). Is there a way to recognize such datapoints?

Can I test your extractor for my device? Unfortunately I'm not a Windows user (not, that's not true ;-)) but I have an old installation of the Viessman Software on a dusty old laptop with Win10.

[gismo2004]

@j0ta29 the tool does currently not query for input/ID --> my values are hardcoded. Will change this to have CLI arguments in future. In addition, the tool should also work on linux as it is a .net core app. You just need to get the DB and translation files from somewhere. In the mean time, i can generate the same files for your device-ID if you like. if you like to play around on your own, the tool is called VExtractor in my repo...

About the div ratio == 0: this was just a stupid mistake in my code :-) div_ratio of 2 seems to be used for values where its accuracy/stepping is 0.5. But i have not tested one of this channels. However its just extracted from DB like this --> see excel. The 3600 div_ratio should match the "Sec2Hour" Conversion of the excel. So quite simple to add.