merbanan / EXOlink

EXOline tcp data parser
2 stars 4 forks source link

Documentation #1

Open aleho70 opened 5 years ago

aleho70 commented 5 years ago

Hello, did you found way how 0x1B in reply should correctly modify XOR checksum? Do you have any other EXOlink protocol description? I found only your code and https://www.swegon.com/globalassets/_product-documents/air-handling-units/gold-version-f/communication/_en/gold_e_exoline.pdf where is description of commands in relation to response value and error codes. Rest information I got only via Wireshark sniffing communication.

merbanan commented 5 years ago

You can decompile the applet. Other then that there is no public available information. If you open the graph view and log out from the applet the data will be transmitted without encryption.

merbanan commented 5 years ago

0x1b is some kind of escape byte.

merbanan commented 5 years ago

I remember from the decompile there where all kinds of strange rules on how to handle data with the escape present.

attah commented 5 years ago

Escape-ersättning Innan ett meddelande skickas görs escape-ersättning. Det innebär att alla före-komster av start-/sluttecken eller escape-tecken i meddelandet ersätts med ett escape-tecken (”1B” hexadecimalt) samt det faktiska tecknet inverterat för att undvika tvetydighet. När ett meddelande sen tas emot görs omvändningen för att återfå det ursprungliga meddelandet.

https://www.msb.se/Upload/Forebyggande/Informationssakerhet/Stod_inom_infosak/foi-r--4438--se.pdf

Hope that helps :) So what is the status on protocol reverse engineering? (I.e. do you have something documentation-ish you could share) I would like to help. Have been planning to do this for some time now, but only now meant to get started properly. Good thing i checked github again.

merbanan commented 5 years ago

Well I haven't written any documentation. But lots of the protocol is understood and that is implemented in the current code. Anyway the info in this document explains one more part.

I'd be glad with some help. The current software works fine for me but the escape handling is lacking.

Current status: All single values from different sensors and status can be read. The unit running status can be read.

Nothing can be set, would be nice to be able to turn off and on the unit. It is not possible to ask for several values at once. Crypto is not supported and not really needed. We might need it to be able to figure out how to turn on and of the units.

Anyway I'm not really happy with the architecture for the c application. Are you willing to put some time into reworking it?

merbanan commented 5 years ago

protocol.txt

Here are some packet captures and some parts of the packet payload structure.

attah commented 5 years ago

Thanks! I'll look at the logs. I'd like to gain some insight into what the commands actually contain, as from what i can tell now they are just opaque blobs. Settings would indeed be nice, and i sure hope they require authentication and/or correct encryption. The paper from MSB says that the protocol is unsafe, although it could primarily be considering confidentiality judging from some of the wording.

I started on an implementation in Erlang because of its awesome bit syntax, which makes it very good for implementing communication protocols. I think i'll do a really simple test server too. Of course this project deserves a good clean C implementation for use in constrained systems. I should be able to help, although i'm a bit rusty on my C. Hopefully further insights and some learnings from my re-implementation will help with direction for refactoring.

attah commented 5 years ago

Here is an attempt at an Erlang client and server https://github.com/attah/exoerl

attah commented 5 years ago

I think i misunderstood the data types a bit, can you verify my new theory? I have my EXOFLOAT down as follows: <<0x3d,Status,Type,Value:32/float-little,Checksum,0x3e>>, which is probably wrong and should be <<0x3d,Length,Type???,Value:32/float-little,Checksum,0x3e>>. The length field would then check out in EXOSHORT too <<0x3d,Length,Value,Checksum,0x3e>>. I see you add the type info... Do you know the client needs to know it, or does my theory on how to infer it look ok? Have you seen any other data types?

merbanan commented 5 years ago

EXOLine-TCP (and maybe EXOLine also) is severely lacking in the message structure department. I mistakenly mapped it to different types. But in reality each query, connection and answer is linked together into one "object". I think there are like 10 different types of queries that returns a specifically formed answer.

The current code was written before I knew better. I didn't have the time to implement things better so I hacked things together a well as I could.

My rule of hacking is that you need to rewrite everything 3 times before it gets usable. So I think we need to design a new libEXOlink api. But before that I think we need a specification that describes the current understanding of the protocol. I'll try to put some hours into that.

merbanan commented 5 years ago

Specification.md is now added. Have a look. I'll try to add descriptions of the stuff the current code supports.

attah commented 5 years ago

I'm not complaining, honestly i don't see what you refer to as a mistake with the data types. Could you please elaborate? I just think did what you did, plus more complete escape handling and assuming length-fields is a thing. Length i'm not certain of, it might just as well be type or status. Do you have thoughts here? Completely agree that this will be revised a lot, hopefully the testserver i made helps with some of the testing. Good start on the spec. It could be worth saying in plain English that escaping is done after calculating the checksum, and unescaping before verifying it, although your example illustrates that very well.

merbanan commented 5 years ago

The mistake I did was assuming that the structure in the response could be parsed on its own. But that is not true, the response payload depends on the query. So you need to couple the query, answer and connection together. In the current code they are de-coupled, which is wrong.

Regarding your comments about the specification, can you send a PR for that?

merbanan commented 5 years ago

I'll try to get some more hours with the specification.

attah commented 5 years ago

Hmm, okay. To me it looks like yo do couple the query and resulting type, at least as float/short... I however don't, but since i have only seen two types i can still tell them apart by length. Or are you thinking abut things like scaling of values? I found one float that is the actual value divided or multiplied with 10 depending if you like percent or values 0-1. Very odd. It was just a very minor comment, but sure.

merbanan commented 5 years ago

switch (cmd->payload_type),

The payload_type is not just a short or a float. It is a complete set of different query classes. Anyway how the current code looks like now is not really vital. The specification will elaborate more how the protocol actually is built. I just need the time to write it.

merbanan commented 4 years ago

I now added handling of escape in the payload.

attah commented 4 years ago

I do agree C is the correct choice, but if you've been toying with the idea of C++, this is likely helpful: https://github.com/attah/bytestream Any plans for more features or further understanding of the requests?

merbanan commented 4 years ago

I don't really have the time and motivation to spend on a proper implementation (in any language). This hack is sort of enough for me and it should be enough as reference for some one else. The only thing missing for me is controlling the fan. But to figure out that I need to implement crypto support, which takes time and motivation which I don't have. So it is what it is and further features might be added but most likely not.

attah commented 4 years ago

Indeed, it does what it is meant to well enough. Control (and alarms) would indeed be neat. Time and effort are indeed precious everywhere. For me any improvements or serious remakes are really contingent on being able to add something useful and/or cool. With lacking understanding of the protocol, that is hard. I should really do some more sniffing.

merbanan commented 4 years ago

I logged in to the java webgui and then went to the graph page and selected one value type to look at. After that I logged out. After logging out the payload was transmitted without encryption. Now sniffing packets was very easy. Figuring out the response payload took a while but now with that done most payloads will be shorts and floats.

jonaz commented 4 years ago

Hi guys, i have a REC indovent FTX which appears to listen to exoline tcp. I tried sending queries but its hard to understand. I have no docs at the moment. Have you figured out how the query works?

I found this

EXOline
EXOline is half duplex protocol. It is of the type Master / Slave and is independent of
physical media. It can be incorporated over the protocols Modbus, BACnet and TCP/IP.
An EXOline query looks like this:
<SoM><EXOline address><OPC><DATA n><CS><EoM>
It starts with a start-of-message token (SoM) and ends with an end-of-message (EoM)
token. It has a command type (OPC) and has a one byte check-sum (CS). DATA is dependant on the specific command. For example if an integer value in the variable storage is
requested, DATA is the variables location. Multiple such commands can be combined into
a so called MulCmd. The recipient is specified using the EXOline address. The recipient
replies with:
<SoA><DATA n><CS><EoM>
where SoA is start-of-answer.

here: http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=8894962&fileOId=8894963

But whats the address? and opc? and how many bytes are the data? Docs i got for modbus on my FTX suggests PLA=254 and ELA=30. Which makes me think i should send something like: 0x3c 0xfe, 0x1e, 0x89.....

according to swegon gold manual 0x89 is read real var.

@attah @merbanan

merbanan commented 4 years ago

Well I never figured out how everything was addressed. I just replayed existing payloads. Does the device answer on the webport with a java applet?

jonaz commented 4 years ago

Well I never figured out how everything was addressed. I just replayed existing payloads. Does the device answer on the webport with a java applet?

unfortunately no web gui deployed on my corrigo. Seems to be an embeeded version developed by REC and it answers 404 on everything i can think of. I have contacted their techs and their devs till get back to me when they are back from vacation in december. Hopefully i can get some info on the web server or exoline protocol.

merbanan commented 4 years ago

Try a wget on ip/EXOApplets.jar

jonaz commented 4 years ago

Try a wget on ip/EXOApplets.jar

Thanks. But 404 there aswell.

jonaz commented 4 years ago

@merbanan do you know the PLA and ELA from your devices webgui or display? i can see it on my display. If so i can probably figure out where the address should be in the protocol :)

merbanan commented 4 years ago

I never reverse engineered it to the point to build my own queries. But I know there are like 10 commands with a fixed set of parameters and a fixed return payload. But even figuring out all that we don't know the internal register address and what it returns. I started documenting stuff but only got this far.

https://github.com/merbanan/EXOlink/blob/master/Specification.md

merbanan commented 4 years ago

Well they answer on this command:

Exhaust air temperature C,ff1ec804b603021be4

merbanan commented 4 years ago

From reading the applet code it looks like you start with 3c and end with 3e.

After 3c you send the pla and then the ela.

Thus my pla is ff and ela is 1e.

merbanan commented 4 years ago

There seems to be a default pla of FF and ela of 1e.

merbanan commented 4 years ago

If we take ff1ec804b603021be4 apart we get the following

ff pla
1e ela
c8 ? some command
jonaz commented 4 years ago

Figured it out :)

send msg: 3c ff 1e c8 04 b6 04 08 00 97 3e
hex:0x3d 0x5 0x0 0xdf 0x56 0xbc 0x41 0x71 0x3e 
decode float: 23.542417526245117

That maps correctly to my current frånluftstemperatur :D

ff 1e = address c8 04 b6 = is probably read real value. 04 08 00 = address of the value to read.

Seems to map to swegon docs aswell since b6 is mentioned there.

I just made 3 for loops which loops over every combination of the address to read to find all available floats. Then i just can check the display to se which matches what temp at the moment :)

merbanan commented 4 years ago

I'm reading the exoline code and i cant match the bytes...

Anyway C8 should be a command, most likely RRL -> Read Register Long maybe. The return from that is a float. But the code only accounts for 3 bytes after the command and we have 6.

So your packet might be this:

 3c som
 ff pla
 1e ela
 c8 cmd (RRL)
 04 ln
 b6 cell
 04 08 00 ???
 97 xor
 3e eom
merbanan commented 4 years ago

Can you link to the swegon docs?

merbanan commented 4 years ago

Ok, I think I know what C8 is, I think that is a multi payload command.

merbanan commented 4 years ago

Ok, b6 matches the command RRP that has the following arguments:

ln cell/60 cell%60

and the command returns a float.

merbanan commented 4 years ago

Command RXP uses the same syntax but returns a byte.

merbanan commented 4 years ago

But the actual list with what is what is not found yet.

merbanan commented 4 years ago

You should be able to find a cell that holds the running state of things also.

merbanan commented 4 years ago

The applet that runs is a generic for all products. To build the interface you use config files, the following files are used:

OverviewAnimation.Acav.gz SetPointLeft.acpt.gz SetPointRight.acpt.gz Alarms.acas.gz Input.acpt.gz Output.acpt.gz Settings.acpt.gz AlarmSettings.acpt.gz TimeSchedule.acpt.gz HolidaySchedule.acpt.gz ManualAutoLeft.acpt.gz ManualAutoRight.acpt.gz RealTimeChart.acrtc.gz

These config files are 16-bit unicode files.

In these files you build the interface somehow.

One key TextMacroFiles = CorrigoEVentilationTexts sets the key/value file to look up strings.

Then all over the place there are keys in a [Cell] or [Row] section. Either the key Visible or Value then contains a string with a value looking like this:

"V,2,1632,R"

This is a comma separated string, it can be 2-4 members long.

Length of 2 has the following parameters: class, ln Length of 3 has the following parameters: class, ln, type Length of 4 has the following parameters: class, ln, cell, type

ln and cell are the values that appear in the transmitted payload. Class describes the method needed to query and type describes the return value.

Types: L - boolean R - float X - 8bit int I - 16 or 32 bit int (not sure what)

merbanan commented 4 years ago

If we open the ManualAutoRight.acpt.gz file. Then look in the CorrigoEVentilationTexts file we find the string 1012 test reference that describe the running state of the Corrigo. It belongs to a Cell segment and has the Value key of "V,3,489,X". This translates to a call to the RXP method with ln=3 and 489/60 -> 8 and 489%60 = 9. The RXP method command is 0x34.

Looking in https://github.com/merbanan/EXOlink/blob/master/exolink.config we find the following payload dump:

0000 3c ff 1e c8 04 [34 03 08 09] 1b e4 3e

This correspond to the string list found in the Cell segment value.

Hopefully the same cell is used to set values in. If so the SXP method is to be used with the method command 0xb0 then the same arguments as RXP but with the addition of an extra byte argument. One byte with the index of the mode values. But it is likely that 489 is a status cell and that the actual toggling occurs in another cell.

attah commented 4 years ago

One Swegon pdf coming up: http://swegon.in/Global/PDFs/Air%20handling%20units/GOLD%20version%20F/Communication/_en/GOLD_E_Exoline.pdf Nice to see some more interest here :)

merbanan commented 4 years ago

Ok, I have now found the data responsible for controlling the ventilation unit. "V,2,801,X"

This ends up calling the SXP method (Set index segment var.) -> B0 ln = 2 -> 02 801/60 -> 0D 801%60 -> 15

Then the gui drop down list correspond to the following string list: "Off,Reduced Speed,Normal Speed,Auto" I assume that it correspond to its index.

Thus to turn off the ventilation unit the following command is issued: FF 1E XX YY B0 02 0D 15

XX is guess to be C8 YY is guess to be 04 or 05

jonaz commented 4 years ago

I have now found all my temperatures in my REC Temovex Blue 4 With the use of a loop looping over 255*255*255 addresses. Its not finished yet... have ben running for 30 hours+.

Extract air send msg: 3cff1ec804b6040800973e hex:0x3d 0x5 0x0 0x58 0x87 0xc6 0x41 0x5d 0x3e decode float: 24.816085815429688

Outdoor air send msg: 3cff1ec804b60404ec773e hex:0x3d 0x5 0x0 0x20 0x93 0x93 0x41 0x64 0x3e decode float: 18.44683837890625

Exhaust air send msg: 3cff1ec804b63b0095353e hex:0x3d 0x5 0x0 0xfa 0xef 0x9c 0x41 0xcd 0x3e decode float: 19.617176055908203

Supply air send msg: 3cff1ec804b63902fa5a3e hex:0x3d 0x5 0x0 0x84 0x70 0xb7 0x41 0x7 0x3e decode float: 22.92993927001953

Set supply air send msg: 3cff1ec804b6040008973e hex:0x3d 0x5 0x0 0x0 0x0 0xb8 0x41 0xfc 0x3e decode float: 23

merbanan commented 4 years ago

After the command (0xb6) 3 bytes are sent. The first byte is the ln parameter. The range of that is unknown but 3b is the max value I have observed. The next 2 bytes are cell/60 and cell%60. Why is that limit used? I think the reason is because 60=0x3C and 3C is a reserved value. I guess they didn't want escaped values in the byte stream if they can avoid it. So byte two and three have a max of 3b and the first byte has an observed max of 3b so my assumption is that the max 3b rule is valid for that byta also.

The above assumption then gives that byte one two and three has the range 00-3c and the total range is 606060=216000. That is 77 times less compared to the complete range.

merbanan commented 4 years ago

And one note this is for one command only. There are like 8 interesting commands.

jonaz commented 4 years ago

My loop over 16581375 variables is now finished and i found 13781 variables which is decodable as floats. (not all seems to be valid but atleat they contain data. Probably some of them are other datatypes since the values are strange)

merbanan commented 4 years ago

@jonaz can you tell in what range the variables where found?

jonaz commented 4 years ago

@jonaz can you tell in what range the variables where found?

I'll do that later this week hopefully. The dump file is about 1 gigabyte :) need to build some tooling to parse it.