Open kbabioch opened 1 year ago
This "API" was entirely created by reverse engeneering. It basially consists of sending magic TCP packets to the heatpump and it returns a shitload of bytes which then can be sepereated into values which then can be interpreted in various ways. I mostly relied on reversing the Android App using a decompiler. The code was far from easily readable but still it was obvious that it was littered with if else constructs that reused some of the values for other purposes if other values had a certain value and so on, a f*** mess, probably historically grown with the requirement of being backwards compatible till the stone age of Altpha Innotec / Novelan / ....
Here you can find some infos about the protocol (all in german though): https://loxwiki.atlassian.net/wiki/spaces/LOX/pages/1533935933/Java+Webinterface
If you want to have a chat, you can send me an email at bouni-luxtronik at owee dot de
Regrding the GLT licence, I know a guy that took the reversing even further and was able to get the root password for ssh into the heatpump and even activate GLT without actually having the licence. But I'll not disclose how to do that for obvious reasons. All I can say is that the GLT interface gives far less info that thr TCP method.
Thank you for all of the work you apparently put into this.
I mostly relied on reversing the Android App using a decompiler.
Seems like you did quite a lot of reverse engineering. Really sad to see that this isn't officially documented, there is basically nothing to loose for the manufacturer and a lot to gain (integrations, etc.).
In any case, the link to the Loxwiki is also interesting (I'm a German so no problem for me).
The code was far from easily readable but still it was obvious that it was littered with if else constructs that reused some of the values for other purposes if other values had a certain value and so on, a f*** mess, probably historically grown with the requirement of being backwards compatible till the stone age of Altpha Innotec / Novelan / ....
The current code is actually quite nice, although there are still many unknowns (not your fault). But the generic structure (from what I can tell), looks quite professional.
If you want to have a chat, you can send me an email at bouni-luxtronik at owee dot de
Will do.
Regrding the GLT licence, I know a guy that took the reversing even further and was able to get the root password for ssh into the heatpump and even activate GLT without actually having the licence. But I'll not disclose how to do that for obvious reasons. All I can say is that the GLT interface gives far less info that thr TCP method.
Well, I also tried some well-known passwords, but they didn't work for me. Definitely interested in logging into my heat pump via SSH, though. Let's discuss this via mail.
By the way: Does this API (TCP packets on port 8889) have some kind of semi-official name? Any obvious strings in the API / decompiled source code that you've worked with?
Since there is also other APIs (websockets, etc.) I'm trying to find out what the differences are and what kind of services are exposed by my heat pump. I can see at least the following open ports:
PORT STATE SERVICE
22/tcp open ssh
80/tcp open http
502/tcp open mbap
8214/tcp open unknown
8889/tcp open ddi-tcp-2
36000/tcp open unknown
22/ssh
and 80/http
are clear, the rest is unknown to me (other than the port used by this library).
So some of the questions that I'm currently looking answers for are:
I'm happy to create some basic documentation to explain this to the best of my knowledge, but so far my knowledge is still very limited ;-).
What kind of APIs services are exposed by my heat pump?
What port is related to which API?
8888 and 8889 are normal (unix?)-sockets which provides the raw values from the internal controller. 8888 is the legacy version. 8214 is the websocket which is used for the web interface on port 80.
What are the differences between the different APIs?
The "socket"-api exposes 3 groups of values. I think they are all the raw values from the heatpump.
For example the flow in and out temperatures are calculations (3004) 10 and 11 in the socket. In the websocket the flow in and out temperatures are in group "Temperaturen (@id=0x0x594240)" with @id 0x0x598308 and 0x0x58d5e0.
How is "heatpump24.com" working, i.e. how is it connecting to my heat pump? (I'm not using it, just want to understand what is technically going on there).
In my case the heatpump pushes every 5 hours the current values to heatpump24.com. Also grep config changes.
Which Android app have you been using for reverse engineering?
For example you can unzip the firmware from download portal. In this is a client web app which connects the websocket. I have tested it with python.
Has anyone contacted the manufacturer to ask for details?
Yes, but I get not much informations.
Thank you for the valuable input @BenPru!
I will try to write a summary, which could be used here (or in your repository) as an overview and starting point for new people joining the project.
8888 and 8889 are normal (unix?)-sockets which provides the raw values from the internal controller. 8888 is the legacy version. 8214 is the websocket which is used for the web interface on port 80.
Do you happen to know what kind of differences there are between those two versions? Websocket sounds nice of course, but I've read somewhere that the legacy API is more powerful?
By the way: Since those services are exposed via network ports, those are not Unix-sockets. Unix domain sockets are basically (pseudo-)files that can be used for interprocess communication. They behave similar to TCP sockets in many aspects, but the main difference is that they are not exposed via network.
See https://serverfault.com/questions/124517/what-is-the-difference-between-unix-sockets-and-tcp-ip-sockets for more details in case you're interested.
The "socket"-api exposes 3 groups of values. I think they are all the raw values from the heatpump.
parameters ID 3003: In "ROM" stored config value which can be changed by customer, technican or the system (e.g. runtime hour counters).
calculations ID 3004: Calculated and not stored values like current temperatures and so on.
visibilities ID 3005: Bool flags to provide informations about activated options and available hardware components.
Thank you again for this summary. Are those terms (parameters
, calculations
, visibilities
) terms that have been extracted from the firmware / app and are "official", or is this what @Bouni came up with? I've struggled in the beginning with those terms and still don't quite like them. It not important once you know what they are about, but still weird terms from my point of view.
For example the flow in and out temperatures are calculations (3004) 10 and 11 in the socket. In the websocket the flow in and out temperatures are in group "Temperaturen (@id=0x0x594240)" with @id 0x0x598308 and 0x0x58d5e0.
So groups can also have IDs? Didn't know that yet. Are those groups similar to the available menu entries on the Luxtronik controller itself?
In my case the heatpump pushes every 5 hours the current values to heatpump24.com. Also grep config changes.
Okay, one more reason to block Internet access for my heat pump :-).
For example you can unzip the firmware from download portal. In this is a client web app which connects the websocket. I have tested it with python.
Cool, will look into this. I've also tested the websocket implementation and was rather shocked about how instable it is. I was able to crash my controller by sending 4 bytes to it.
Maybe you can test/verify this:
#! /usr/bin/env python3
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM);
client.connect(("192.168.1.5", 8214));
client.send("Lux_WS".encode());
For me this halts the controller. After about 5 seconds, it gets rebooted (via a Watchdog or so). I've also recorded a video about this: https://www.youtube.com/watch?v=lhcy8SzfIxY
I've already reported this to Alpha Innotec, hopefully they are working on fixing it. Would be interesting to know if others are also affected by it, though.
Yes, but I get not much informations.
Well, haven't asked specific technical questions, but they sounded helpful to me. Do we have a list of essential questions that we want to know more about?
Do you happen to know what kind of differences there are between those two versions?
No.
Websocket sounds nice of course, but I've read somewhere that the legacy API is more powerful?
Yes.
Are those terms (parameters, calculations, visibilities) terms that have been extracted from the firmware / app and are "official", or is this what @Bouni came up with?
They are not official. parameters have two "ids" one for read and one for write.
So groups can also have IDs? Didn't know that yet.
In the Websocket groups have ids. But all ids are temporarly! And the Names are hardcoded in german I think. So you can't match anyhing.
Are those groups similar to the available menu entries on the Luxtronik controller itself?
No.
After about 5 seconds, it gets rebooted
Last week I have updated from 2.86.0 to 2.88.1 and the new version seems to be not so stable. Ait is bugfixing. I think the reconnect on every command is a problem. I have changed this and after more tests I will push it as pr: https://github.com/BenPru/python-luxtronik/blob/Optimizing-data-reques/luxtronik/__init__.py
Do we have a list of essential questions that we want to know more about?
We use a internal api. 8889 can used without passwort or security. You can write all values. I don't think they support the internal api.
Thank you for your findings.
Last week I have updated from 2.86.0 to 2.88.1 and the new version seems to be not so stable. Ait is bugfixing.
Do you know if the Luxtronik controllers can also be downgraded, or are you know stuck with 2.88.1?
By the way: Support told me that they're currently working on a new version with new features (Energy monitoring), which is supposed to be released soon.
I think the reconnect on every command is a problem. I have changed this and after more tests I will push it as pr: https://github.com/BenPru/python-luxtronik/blob/Optimizing-data-reques/luxtronik/__init__.py
Will you try to get this integrated here? Is your repository just a fork with the intention to get changes integrated back here, or do you plan to run your own fork?
We use a internal api. 8889 can used without passwort or security. You can write all values. I don't think they support the internal api.
That sounds bad. We should probably try to report this.
Have you ever had a look at the other open ports?
# netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:36000 0.0.0.0:* LISTEN 370/appl
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 354/httpd
tcp 0 0 0.0.0.0:502 0.0.0.0:* LISTEN 370/appl
tcp 0 0 0.0.0.0:8214 0.0.0.0:* LISTEN 370/appl
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 350/dropbear
tcp 0 0 0.0.0.0:8889 0.0.0.0:* LISTEN 370/appl
netstat: /proc/net/tcp6: No such file or directory
udp 0 0 0.0.0.0:46346 0.0.0.0:* 363/appl
udp 0 0 0.0.0.0:4444 0.0.0.0:* 368/appl
udp 0 0 127.0.0.1:5002 0.0.0.0:* 358/bootmgr
udp 0 0 0.0.0.0:502 0.0.0.0:* 370/appl
netstat: /proc/net/udp6: No such file or directory
I don't think they support the internal api.
Probably not, but we could still ask and see what they will reply. I have quite a good and respectful person of contact.
Do you know if the Luxtronik controllers can also be downgraded, or are you know stuck with 2.88.1?
Yes, the support says the downgrade is possible. But I think I stay on 2.88.1. I have optimize the connection on request site.
Energy monitoring
Yes, 2.88.1 has energy monitoring features: see thread https://github.com/BenPru/luxtronik/issues/59#issuecomment-1369405019
Will you try to get this integrated here?
I have added two sensors for this domestic_water_energy_input
and heat_energy_input
. See pre-release 2023.01.07
Is your repository just a fork with the intention to get changes integrated back here, or do you plan to run your own fork?
Currently I'm trying to extend BenPru/luxtronik and provide it as ha core integration. See repo https://github.com/BenPru/core-1/tree/luxtronik. PR in the ha core main are only reviewed with single ha domains (climate, sensor) so I plan to push only one domain in the first step. I will create another branch there or copy it to a branch in BenPru/luxtronik and provide it with all domain sensor for tests with other users.
Have you ever had a look at the other open ports?
No, just the websocket port 8214. See python test.
Well, haven't asked specific technical questions, but they sounded helpful to me. Do we have a list of essential questions that we want to know more about? Probably not, but we could still ask and see what they will reply. I have quite a good and respectful person of contact.
ad hoc questions:
circulation_pump_heating
(calculations.ID_WEB_HUPout
) runs very often. Sometimes about 18 hours/day without a heating/water requests for days and outdoor temp about 10 C° (no frost protect needed). Why? How can I control this?Yes, the support says the downgrade is possible. But I think I stay on 2.88.1. I have optimize the connection on request site.
That sounds good.
I have added two sensors for this
domestic_water_energy_input
andheat_energy_input
.
Cool, wasn't aware of that yet.
Short overview of my lovelace view:
Is this what you get by default, or after (heavily) customizing your integration? The amount of data looks pretty interesting.
I have to create unit tests and so on. Interested in helping?
Yes, definitely, that's also what I'm very familiar with. Happy to provide some infrastructure for unit testing Python code.
How can I start TDI manually "now"?
I would also like to know that. If you figure it out, let me know :-).
The circulation_pump_heating (calculations.ID_WEB_HUPout) runs very often. Sometimes about 18 hours/day without a heating/water requests for days and outdoor temp about 10 C° (no frost protect needed). Why? How can I control this? Can we listen for events (websocket or socket) to change the handling from pull to push? Push value changes / callbacks #38 Websocket: How can we unique identify the values? (name is short in german and ids change every request) How to request the firmware revision over the socket? Changelog V2.88.1
Good questions, unfortunately I cannot answer them.
I have de-compiled the application by now and could try to figure out some of the details from the binary running on the controller. However most of this is really time-consuming, and so far that's my biggest limitation :-).
Is this what you get by default, or after (heavily) customizing your integration?
The integration has grown over time. The current version of my integration creates the most of it out of the box.
@kbabioch : Which decompiler do you use? I just have disassembled the firmware, but reading assembler is....
@BenPru : Do you think that circulation_pump_heating (calculations.ID_WEB_HUPout) is a electrical heating of the circulation pump? I would guess that this only means that the circulation pump (for the heat circuit) is running, i.e., "HUP" stands for "Heizungsumwälzpumpe" and "out" means that this is an output of the heat pump. This could be the "HUP" listed under "Ausgänge" in the websocket interface.
@BenPru : Do you think that circulation_pump_heating (calculations.ID_WEB_HUPout) is a electrical heating of the circulation pump?
No. It is the heating circulation pump. I think this is a mistake. It should be heating_circulation_pump.
Concerning "custom" TDI time
By looking at the firmware, I don't think that it is possible to say that the TDI should start now. It can only start at midnight. One workaround would be to change the clock (or the time zone?) of the heatpump such that midnight happens when the sun is shining (e.g. PV energy is available).
However, I think there is a more reasonable approach. There is a parameter which allows the heat pump to use the heating rod (ZWE) if your requested warm water temperature is higher than what the inverter can accomplish (I did not yet checked the parameter number, it can be found under "Einstellungen" -> "Warmw. Nachheizung"). This is documented in the manual by AIT. Then, when you have PV energy you could raise the warm water temperature (parameter 2) to, let's say, 65°C. Then, the inverter heats until, e.g., 55°C and the heating rod should finish off. (Maybe one should also alter the hysteresis parameter 74). In theory, this should be roughly the same as using TDI. I did not try this approach, but I get PV in "the near future" and would like to try it.
@kbabioch : Which decompiler do you use? I just have disassembled the firmware, but reading assembler is....
ghidra. It de-compiles most of the appl
binary just fine, although there are sections that are not properly analyzed and it's really hard to understand (menues on the controller).
I'm not too familiar with ARM assembler, so anything that is not de-compiled is tricky for me.
I would guess that this only means that the circulation pump (for the heat circuit) is running, i.e., "HUP" stands for "Heizungsumwälzpumpe" and "out" means that this is an output of the heat pump. This could be the "HUP" listed under "Ausgänge" in the websocket interface.
No. It is the heating circulation pump. I think this is a mistake. It should be heating_circulation_pump.
Sounds to me that you're talking about the same thing here?
By looking at the firmware, I don't think that it is possible to say that the TDI should start now. It can only start at midnight. One workaround would be to change the clock (or the time zone?) of the heatpump such that midnight happens when the sun is shining (e.g. PV energy is available).
That's a pitty.
Then, the inverter heats until, e.g., 55°C and the heating rod should finish off.
The problem with that approach is that it will turn on the inverter / compressor, which means there is physical stress to a component that has only limited amount of life time. I would rather prevent this from happening, and use the heating rod to heat up my water without wear and tear to the system.
In my case I'm not even wasting energy, since my PV system needs to limit itself during peak hours ("70% Regelung"), so I would rather heat up my water than to not use the excess energy.
Maybe I should ask the support, if they would be willing to add such a feature as Kurzprogramm
or something like this. The energy management capabilities are rather limited currently.
Then, the inverter heats until, e.g., 55°C and the heating rod should finish off.
The problem with that approach is that it will turn on the inverter / compressor, which means there is physical stress to a component that has only limited amount of life time. I would rather prevent this from happening, and use the heating rod to heat up my water without wear and tear to the system.
Yes, but currently, TDI is doing the same thing? And in summer, when you only need warm water, you also start the inverter just for heating your water. I do not see why using the heating rod afterwards means additional stress to the system.
Yes, but currently, TDI is doing the same thing? And in summer, when you only need warm water, you also start the inverter just for heating your water. I do not see why using the heating rod afterwards means additional stress to the system.
Yes, currently that's the case. Would prefer to easily enable / disable the heating rod, though. I would like to implement some logic to turn on the heating rod, whenever there is an PV excess for more than a couple of minutes and when the energy would be "wasted" otherwise. I don't think I want to turn on the compressor on and off all of the time.
Currently I'm just looking into the technical feasibility. Unfortunately it seems that what I have in mind is not easily possible. That's unfortunate. Thank you for looking into it, anyway.
Yes, but currently, TDI is doing the same thing? And in summer, when you only need warm water, you also start the inverter just for heating your water. I do not see why using the heating rod afterwards means additional stress to the system.
Yes, currently that's the case. Would prefer to easily enable / disable the heating rod, though. I would like to implement some logic to turn on the heating rod, whenever there is an PV excess for more than a couple of minutes and when the energy would be "wasted" otherwise. I don't think I want to turn on the compressor on and off all of the time.
Currently I'm just looking into the technical feasibility. Unfortunately it seems that what I have in mind is not easily possible. That's unfortunate. Thank you for looking into it, anyway.
Looks like the discussion on TDI is taking place in 2 separate threads. Have a look at my reply in the other thread: https://github.com/BenPru/luxtronik/issues/4#issuecomment-1383997313
Yes, I didn't know about the other thread. Thank you!
Can anyone extract the names of the new values in V.88.? Parameter 1126-1140 1136 is heating_energy_input and 1137 is domestic_water_energy_input Calculation 260-262 Visibility 355-364
As far as I remembered I extracted the names out of the Android App. Not sure if they are in the pump firmware itself
I haven't found a comprehensive list of names / values for parameters yet. Only some references to getParameter
and storeParameter
next to variable names when they are used.
@gerw Any more luck with this yet?
Anyone (still) able to de-compile the Android app?
No, I also do not see how to make the connection between the parameter names and their number.
I decompiled the Android App but couldn't find any sort of list containing any names. To be honest I do net even remember where exactly I found the original list of names 🤔
Maybe the list is in the java-application of the old webinterface? (I do not have the java application since my heat pump is too new)
Btw.:
V3.85.6
while others here report something like V2.88.1
. Thus, I guess that my 3
is related to the softwareID
on heatpump24.com (although my heat pump has softwareID=2
). Then, the 85.3
or 88.1
is the "real" version.Then, the 85.3 or 88.1 is the \"real\" version.
Yes. V1, V2, V3 ist the heatpump Type. See download Portal https://github.com/BenPru/luxtronik/blob/main/custom_components/luxtronik/update.py#L119
Is there a changelog available for the different firmware versions?
As far as I know, no there is no official changelog. Talked to the support regarding this, they plan to do something like this in the future, since some people have been asking for it.
Maybe if all of you could contact them regarding this (AIT DE Customer Support Center <support@ait-deutschland.eu>
) it gets more priority. Releasing binaries without a changelog is really not good practice. I would like to know which parts of the code have been changed / modified / fixed, and what I need to look out for.
official changelog - since some people have been asking for it.
Guilty 👀 - Asking for a friend. 🤣
@gerw Any more luck with this yet?
The coupling between the parameter numbers and the names is done in the method _Z15Parameteraufrufiii
. After a new parameter value has been set in the parameter array ParamCollection
, this function is called and saves the parameter value a second time in a named global variable. For some parameters, some small, additional logic is performed.
In this method, there was presumably a switch
statement with (for my firmware version) 1950(!) cases. Ghidra is not able to decompile it. It is possible, if one restricts to the first 1000 statements, but I did not found anything new, only the following observations:
DHW_CoverageOfHeatPump
, where HW
seems to refer to hot water
The coupling between the parameter numbers and the names is done in the method
_Z15Parameteraufrufiii
Cool, thank you for your analysis.
In this method, there was presumably a
switch
statement with (for my firmware version) 1950(!) cases.
Yes, that sounds messy. Probably it will be difficult to reverse engineer all of that without having the sources available. I guess most of the developers don't even know all of the details about those parameters :-).
After a new parameter value has been set in the parameter array ParamCollection, this function is called and saves the parameter value a second time in a named global variable. For some parameters, some small, additional logic is performed.
Do you understand why there are parameters and global variables? I guess parameters are read somehow from non-volatile storage (there are some param files in /home) during initialization. If I were to develop this, I would create some kind of struct (ParamCollection
) and keep it in memory and only sync it back to non-volatile storage when values are changing and/or during some kind of save operation. I don't see why individual global variables would be needed, all could be accessed via a global pointer to the ParamCollection
struct in memory. But I guess they did it differently.
parameter 2 is now called DHW_CoverageOfHeatPump, where HW seems to refer to hot water
In our codebase parameter 2 is:
What does the change mean? Has the meaning of the variable been changed? Or only the name? Which firmware version are you working with?
parameters 8, 19, 30, 31 (and many more) seem to be no longer in use, at least they are not assigned to a named variable.
Hm, in our code base those are Unknown parameters with some IDs / names extracted from somehwere. If parameters are being re-used / changed / no longer used, it will be very difficult to understand them and keep track of this in the code.
https://github.com/Bouni/python-luxtronik/blob/973f7c98167fa837f316e63d73aecebf5a2f1785/luxtronik/parameters.py#L37 https://github.com/Bouni/python-luxtronik/blob/973f7c98167fa837f316e63d73aecebf5a2f1785/luxtronik/parameters.py#L48 https://github.com/Bouni/python-luxtronik/blob/973f7c98167fa837f316e63d73aecebf5a2f1785/luxtronik/parameters.py#L59-L60
At least for 30
and 31
I would expect that this functionality is implemented somewhere somehow, since I can see the counter for Abschaltungen
and average runtime in the menu. But who knows (other than AIT) where it is currently being managed / stored :-).
I am currently working with version V3.85.6, since that version is running on my heat pump.
I do not have any precise idea, why we have global variables and the variable array. Maybe, because Einst_Zugangscode
is easier to read than ParamCollection[107]
?
And yes, the parameters are saved to the files appl_param1
and appl_param2
. Therein, each parameter occupies 8 bytes: 4 bytes for a flag and 4 bytes for the parameter itself.
Concerning parameter 2: I don't think that the meaning has changed. I think that this is just the temperature for the warm water which was demanded and which was achieved by the heat pump.
Parameters 30 and 31: Which menu do you mean? In the code, there is no reference to variable names containing switchoff
. The parameter 30 seems to be always 0
for me, but 31 takes the values 1
and 4
...?
average runtime
I think this is calulated on client side: calculations.ID_WEB_Zaehler_BetrZeitWP / calculations.ID_WEB_Zaehler_BetrZeitImpVD1. I can't find my displayed value in any parameter or calculation.
And yes, the parameters are saved to the files
appl_param1
andappl_param2
. Therein, each parameter occupies 8 bytes: 4 bytes for a flag and 4 bytes for the parameter itself.
Okay, so the values itself are 4 bytes. Makes sense, was wondering myself why 8 bytes are occupied when the values could be easily represented in less bytes.
Do you know anything about those flags? Are they "interesting"?
Concerning parameter 2: I don't think that the meaning has changed. I think that this is just the temperature for the warm water which was demanded and which was achieved by the heat pump.
Ah, okay, then I mis-understood it. Because the name you've mentioned was different from what the library is referring to this parameter.
Parameters 30 and 31: Which menu do you mean? In the code, there is no reference to variable names containing switchoff. The parameter 30 seems to be always 0 for me, but 31 takes the values 1 and 4...?
There is a menu on controller and also in the web interface, which displays the average runtime. I think its part of the Betriebsstunden
. There you can see the total number of operating hours (heating vs. hot water) and also the number of starts of the compressor. For physical reasons you want to have a long average runtime, so that value is interesting. I was expecting this to be a parameter / calculation, but @BenPru could be right and this might be calculcated based on the other two values. Haven't had a deeper look into the firmware with this in mind.
This project is helpful and I was able to read off some parameters from my heat pump system. However I'm missing some kind of meta-information about this project and the API it uses.
Is there any kind of information about this API, or has all of this been reversed engineered? Is there anyone that can be contacted to answer some questions?
By the way: I also have the GLT license available, so I could double-check some aspects using the "official" Modbus API for instance.