Support for Cecotec Conga 4690

Olliek95 commented 3 years ago

Hi!

You asked me to open an issue for keeping track of the support for the Conga 4690 from Cecotec. So here it is. 😄 What's the current status of the support regarding the Conga 4690? I'm a software engineer myself so I wouldn't mind contributing to the support for this particular version of the protocol. 👍🏻 If there is anything I can do to help, please let me know.

Thanks!

adrigzr commented 3 years ago

Hi @Olliek95,

The current status is 0% :) but I don't mind adding support for it as long as I have enough information about the protocol. Unfortunately, I don't have access to a 4690 model so I need to someone to gather information for me.

To begin with, I need to have access to some raw tcpdump to try to understand the new protocol. The problem with this model is that it works by using secure communications, so we need to bypass it.

The first approach could be to try to switch SSL to off in your robot by editing the following files:

# /mnt/UDISK/config/sysConfig.ini
# /etc/sysconf/sysConfig.ini
- server_ssl_enable=1
+ server_ssl_enable=0

Next, restore whatever change you did in the hosts file (just remove the valetudo redirection) and do a full restart:

reboot

At this point, try to open the official app and see if the robot connects to the cloud. I'm 99% sure that it won't work, so you need to sniff the protocol using SSL and this is hard.

Restore the sysConfig.ini lines and try to record a dump using tcpdump and read it using wireshark with TLS.

To record a dump I used to use the following:

tcpdump -i wlan0 -s0 -w - 'tcp and not port 22' > /tmp/dump.pcap

Then, restart the robot app:

kill $(pidof RobotApp)

Then, open the official app and do some operations. After that, stop the dump with <Ctrl-C>. You can transfer the dump file to your machine with scp and inspect it with wireshark.

Note: you can try to record a dump with SSL turned off to see how it would look like if you decode the SSL protocol correctly.

Good luck with your endeavors :)

Olliek95 commented 3 years ago

Hi!

Thanks for all the information. I've been trying some stuff, so here is a small update.

DIsabling SSL in the configuration file you mentioned indeed results in the app not working anymore. So it seems like the server is not responding / handling the non-SSL traffic from the Conga 4690 properly. It does however seem like the robot itself is trying to use the non-SSL ports to communicate with the server (so maybe this is a server problem and not so much the robot-software issue).

Because of this I decided to leave the SSL disabled and making sure the robot communicates with my Valetudo server running on my home server in the local netwerk. After restarting the application on the robot (a full reboot is not required by the way) and restarting the Valetudo home assistant addon, I noticed the Valetudo home assistant addon is "kind-of" responding to the robot. WIth responding, I mean throwing an exception, that does seem to be a result of some sort of communication from the Conga. To be precise, I received the following error from the Valetudo home assistant addon:

[2021-08-29T20:34:10.719Z] [ERROR] Uncaught Exception { err: DomainException: Max length exceeded at PacketSocket.onReadable (/snapshot/app/node_modules/@agnoc/core/lib/sockets/packet.socket.js:110:29) at processImmediate (node:internal/timers:464:21) { metadata: undefined }, origin: 'uncaughtException' }

I'm not quite sure what the cause of this exception is, but could it be possible that the non-SSL communication from the Conga 4690 could be very similar (or might actually be similar) to the other Conga robots in this serie? I wouldn't mind leaving SSL disabled if this could result in a working Valetudo setup.

By the way, this is the network activity after restaring the robot application on the Conga (192.168.0.10 is the server running the Valetudo server and 192.168.0.118 is the Conga): Could it be that the POST requests results in the "max length exceeded" error above?

Does this information provide you with some insight or do you need more information?

Thanks in advance!

adrigzr commented 3 years ago

Hi @Olliek95,

It seems like the server is not allowing non-SSL connections but the robot supports it. It is good to know this.

Unfortunately, it looks like the protocol is different, so we need to figure out how does it works. The problem here is to RE the protocol with the SSL enabled.

So, first of all, restore the SSL communications and set this environment variable in your session:

export SSLKEYLOGFILE=/tmp/key.log

Then, restart the server (RobotApp) again to see if the SSL is being dumped into the file. If it works, try to record some traces and open them using wireshark as described here. If it doesn't you need to try a different approach like doing some man-in-the-middle to extract the protocol. By your screenshot it seems like it uses HTTPS protocol now.

With a full dump file and a valid SSL key I think I will be able to RE the protocol.

Regards,

Olliek95 commented 3 years ago

Hi @adrigzr ,

Sorry for the late response.

I've been able to successfully retrieve the SSL in the dump file and have imported it into Wireshark. After performing some actions, it seems like only the very first encrypted message is decryptable using the dumped SSL, as can be seen in the image below:

I noticed various "Change Cipher Spec" messages in the logs as well:

Could it be that the SSL are been changed regularly to prevent the same SSL key from being used multiple times?

Thanks!

adrigzr commented 3 years ago

Hi @Olliek95,

Don't worry about the rush. Take your time, RE is not easy :)

Thats are great news! The next step is run a dump, kill the RobotApp and record some actions like "Locate Robot" using the official app. Then we can work on understanding the protocol using it.

About the "Change Cipher" thing idk. I've never tried to RE an SSL protocol before.

After you record the full dump with some robot commands, send it to me privately and I'll take a look to see if there is any similarity with other robot protocol.

Regards,

Olliek95 commented 3 years ago

Hi @adrigzr,

I actually already made a full dump after killing the RobotApp and performing some actions afterwards (including "Locate robot"). The problem though is that it seems like the SSL key recorded in the key.log file can only be used to decrypt a fixed number of messages, after which it seems to change (without it being recorded in the key.log file). All messages up to the first "Change Cipher Spec" message, seem to be decryptable. This is why you can see multiple data packets in the first image I posted in my last post but only the first I was able to decrypt this way. This means that basically only a few data packets are decrypted by Wireshark, the remaining packets remain encrypted and are probably unusable to you.

I'll see if I can figure out a solution for this and will come back to you. If you have any suggestions to overcome this issue, please let me know. :)

Olliek95 commented 3 years ago

Another update, unfortunately no breakthrough yet.

Did some more testing with the SSL key dump. It seems like after the first few messges, the robot app starts another TLS session with the Cecotec server for port 9090 (instead of 8001), which uses a different randomly-generated SSL key: And for some reason, only the first SSL key is being saved to the log file specified by the SSLKEYLOGFILE variable... This explains why only the data for port 8001 can be decoded (see green rows above), since the data over port 9090 is using a different encryption key.

Until this I've found no way to export the additional second key used for the 9090 communication yet, which is a shame (since the interesting application data seems to be transmitted over port 9090). I've been looking all over the web for more information regarding this, and a possible reason why only the first key would get exported to the log file, but nothing so far...

I've also tried adding the EXPORT for the SSLKEYLOGFILE variable to /etc/profile to set it at boot time so that all processes can use it, but this doesn't seem to be working. The robot app is running fine but nothing is being written to the key log file (which seems a bit odd to me). When running "echo $SSLKEYLOGFILE", the correct path is shown. Also when running the robot app manually through SSH does write data to the log file.

Do you happen to have any ideas what could be the problem here?

adrigzr commented 3 years ago

Hi @Olliek95,

I'm not an SSL expert but I think the SSLKEYLOGFILE variable is kind of limited. The only way I can think of for RE the communications is doing a MitM.

There are several approaches for this. You can setup a computer to act as a proxy and configure it in the robot network, this is the easier one.

If this doesn't work, you can setup the computer to act as a middleware between the communications with the cecotec cloud. To do this, you need to redirect all the traffic (with the hosts file) to your computer, log it and then connect to the official servers and send the data to them. For this you need to configure a secure AP in your computer and set the public key as a known one in the robot.

With both approaches, you can now log all the traffic in clear so we can work with it.

Keep up the hard work! You are so close :)

Regards,

morbith-dqtz commented 2 years ago

Hi @adrigzr,

I'm also interested into port conga 6090 to agnoc, and I found that it behaves in the exact same way.

I also found that It's easy to decode all the traffic using mitmproxy and a couple iptables rules in the Access Point to forwards the outgoing connections from conga to 8001 and 9090 ports to mitmproxy.

This way its trivial to read all HTTP/WSS exchange in plain text.

iptables -t nat -A PREROUTING -s conga.ip -d 0.0.0.0/0 -p tcp -m tcp --dport 9090 -j DNAT --to-destination mitm.ip:8080 iptables -t nat -A PREROUTING -s conga.ip -d 0.0.0.0/0 -p tcp -m tcp --dport 8001 -j DNAT --to-destination mitm.ip:8080

This kind of protocol also uses an additional host named tcp-cecotec witch handles this WSS commands at port 9090, it's defined by a JSON response crafted by the cecotec-ota host

There are some protocol documentation to look at and try to help you at some point ?

Thanks a lot for your time and work!

adrigzr commented 2 years ago

Hi @morbith-dqtz,

Those are some good news! Unfortunately, idk if there is any protocol documentation for this kind of communication.

For me the starting point would be to try to create a "mock" server imitating the communication with the retail server and the robot using the host file to redirect all the robot traffic to the mock server.

I'd try to use non-ssl communication for this mode because it is much easier than trying to add a custom SSL cert to the robot.

Once you have a basic mock server working, I could try to add the code to this lib and add compatibility with Valetudo.

Regards,

Yhatax commented 2 years ago

I'm stuck without connection on my 6090, so until that is fixed by Cecotec I can't be of any help. But once I can be sure I'll get that mitmproxy running to know whats going on.

morbith-dqtz commented 2 years ago

Hi @adrigzr,

At this point I'm able to send conga 6090 to dock, do manual moves, read the status (battery, watter lvl ...) and I haven't got implemented but Its easy to achieve the rest of necessary interactions, get maps, clean plans, etc ...

All without SSL.

The bad news are that at this point i'm not sure if agnoc is the right way to support this kind of robots.

To achieve this goals I use a webserver to send a JSON config to the robot. Then the rest of the interactions are made via a websocket server. I made some grep to agnoc sources and I don't see that it uses websockets at all.

For example, to move the robot I send a message like :

{ "tag" : "sweeper-transmit/to_bind", "content" : "{ \"control\" : \"set_direct\", \"direction\" : 1, \"angle \" : 0.0 }" }

The communication is made by the exchange of this type of ws messages. The protocol uses a pseudo login process that needs to be aswered, then the ws send a "subscription" message to the robot and it starts sending messages to the ws server with status and start accepting the rest commands.

adrigzr commented 2 years ago

Hi @morbith-dqtz,

Nice progress right there! Keep up the good work!

agnoc was made as an agnostic lib to support every Conga robot in the market. So, we can blend it to support any protocol out there. We just need to be able to recognize the model based on how a robot is connected, via raw-tcp communication or http and websockets. We should be able to use its API to communicate with the connected robot and leave the protocol as an implementation detail.

Where can I see your progress with the protocol?

Regards,

morbith-dqtz commented 2 years ago

I used a very simple WS server written in python to start digging into the protocol. After this steps I found that it's too much simple and the library don't support binary messages, so I couldn't play with maps and so on.

I'm planning use node websocket server for the next steps, but I need find a good starting point for thee WS schema to continue.

At this moment all lab testing and devel is in my local environment under constant rethinking :P

adrigzr commented 2 years ago

@morbith-dqtz, don't worry about that. Prototyping is the way to go when you are REing a protocol :)

morbith-dqtz commented 2 years ago

Hi @adrigzr

I have written a minimal but functional ws server in node.

I'm able to send commands to de robot via ws client and I have some insights of the protocol. I found that the the CongaCli APP is not accurate sending that control messages.

For example :

As i told before Conga APP sends the following message to proceed with a manual move :

{ "tag" : "sweeper-transmit/to_bind", "content" : "{ "control" : "set_direct", "direction" : 1, "angle " : 0.0 }" }

But the handler function that calls everest manager only cares about direction.

After understand the logic behind function arguments that robot takes from de JSON message, I test my assumption and I send :

{ "tag" : "sweeper-transmit/to_bind", "content" : "{ "control" : "set_direct", "direction" : 1 }" }

And works as expected.

So ... sniffing the conga app probably is not the best way to proceed in the REing it's protocol xD

txitxo0 commented 2 years ago

Maybe they are using the same app (commands) for multiple robots and others accept this json property... But yes, at least for this robot....

Yhatax commented 2 years ago

Your progress is awesome, Conga APP is in many ways a malware XD. I guess you are on the right direction already, having all interactions with beetween the robot and the app captured in the proxy and transfering then to your ws server.

Maybe the angle is used when map has been rotated, and is sent because map is rotated in the app. I see great news when you share any progress.

morbith-dqtz commented 2 years ago

Hi @Yhatax ,

I'm not redirecting traffic, first the robot asks to a webserver where is his WS, and I tell him to connect directly to my WS :)

Then all commands are passed to the robot via this WS using a python cli connected to the same WS

Yhatax commented 2 years ago

With transfer I ment you first capture the interaction (so you have a rough idea of how the action is asked and executed, I'm stuck here because my app is not working), and then RE it to add it ("transfer") to the ws server, so the robot connects to it and the server sends the corresponding command from local network to the robot,

Anyways, great work! I'm willing to get mine back to work XD.

morbith-dqtz commented 2 years ago

Hi @adrigzr

I've written some tools that allow me to interact with the robot using the native cecotec application.

https://github.com/morbith-dqtz/agnoc-tools

It's not Valetudo, but permits to operate the robot with the APP and use only the local services. I can edit maps, create/edit clean plans, move manual, get info about consumables ... It's far to be complete but I think in it's state you can figure out how the whole thing works and a way to integrate with agnoc driver.

I'm not sure valetudo can parse the 6090s maps, I read some things about this topic and seems its not supported at this moment.

I started playing with the Congas RF subsystem, It seems possible to replicate and extend the congas 6090 remote :) at least RF REC/PLAY commands works fine :P

adrigzr commented 2 years ago

Nice work @morbith-dqtz,

I'll take a look at it asap. In the meantime, could you elaborate a diagram about how the protocol works? I'm not sure about understanding it completely. You can use this tool: https://swimlanes.io/

It seems that you achieve connecting to your local server using TLS. Are the certs autogenerated and injected to the robot? How does it work?

If the robot has a remote, you can use a broadlink to record the commands and connect it directly to HA. On the other hand, you wont get map support I guess.

Regards,

Yhatax commented 2 years ago

As far as I understand, ssl has to be disabled and then, commands are sent to the robot.

Great work @morbith-dqzt this will help a lot with local usage. It would be nice to test it for 4690, and see if there is any diference.

morbith-dqtz commented 2 years ago

Hi,

@adrigzr I'll try to elaborate a diagram when I have some time.

I used bind to handle all the domain requests, I also generated a key used as CA that signs a request certificate from another key, then I used that cert, signed by my CA, in the webserver at 8001. Is not necessary to injecting certs anywhere, They only check if it's a self-signed cert ...

If you control that first TLS web step you can configure APP & Robot as you wish.

@Yhatax Is not necessary to disable SSL in the robot. When the robot gets that first JSON configuration, it's enough point to ws:// rather than wss://

Olliek95 commented 2 years ago

Hi all, any update on this by any chance? :)