search

Fusseldieb / ttlock-reverse-engineering

5 stars 4 forks source link

State of encryption #1

Open kind3r opened 3 years ago

kind3r commented 3 years ago

Hi @Fusseldieb,

After impulse buying a smartlock, I realized that it can be used in the ttlock ecosystem, which got me intrigued. So I did some reasearch to find out more about this 'open' protocol and to check if I can integrate it with my Home Assistant. Long story short, I got to this repo which basically sums up what I would like to get from this lock (remote locking and unlocking without or Chinese friends knowing about my habbits, or worse, being able to remote unlock my door).

Now, I started diging up through the code, but because I used an older version of the ttlock sdk that I found in the react-native repo (actually there is a npm react-native-ttlock package that contains it) I was able to get a bit further (maybe?) in my quest, as this version of the SDK decompiles much cleaner so it's easier to follow the code. The JNI libLockCore.so seems to be the same, and it's functionality is very basic as you already pointed out (there is a nice tutorial about decompiling with Ghidra here) so I have replicated the functionality in a small node.js lib. I have also build a small Android app that includes the JNI lib to check the results agains my js code.

Now, about your issue with not being able to decrypt the data part of the package.

  1. The last byte in the data part of the package is the checksum, so you need to remove that, so you are left with 15 bytes
  2. Based on your packets header (protocol 5, subversion 3) the lock type is 5 which means that the data is decrypted with AESUtil::aesDecrypt(), so you need to get the aesKeyArray from the lock, after you are able to pair with it (as far as I can tell afer 2 days of digging through the java code).

I don't have the lock yet, so I can't test my theory. Also my experience with Android and Python is very limited and I prefer JS for the ease of use. But since the SDK does not seem to phone home it could be used in a standalone app for pairing, initial setup and credential management, and then have a separate HA component that uses those credentials issued by the app.

Fusseldieb commented 3 years ago

Thanks for the extensive explanation.

What you've discovered is awesome already, can't wait to see more.

I don't have the lock yet, so I can't test my theory.

Meanwhile, if you want, I can replicate some things for you on my lock, if that brings you further.

Also my experience with Android and Python is very limited and I prefer JS for the ease of use.

I'm comfortable with Node and Python, so that shouldn't be an issue per se.

Now I'm tempted to try more... You mentioned some interesting parts there! :)

2. Based on your packets header (protocol 5, subversion 3) the lock type is 5 which means that the data is decrypted with AESUtil::aesDecrypt(), so you need to get the aesKeyArray from the lock, after you are able to pair with it (as far as I can tell afer 2 days of digging through the java code).

That helps a lot! Do you have any pointers on how to get the aesKeyArray from the lock? Do you think I pair it with my phone and it sends it still in "cleartext"?

  1. The last byte in the data part of the package is the checksum, so you need to remove that, so you are left with 15 bytes

But isn't the data part encoded/encrypted? Could you give me a more practical example?

I could get the hci bluetooth dump from my phone after pairing the lock with my phone, if it interests you.

kind3r commented 3 years ago

I'm still struggling with the BLE code that gives different results on scans on my mac and linux, still learning how it all works. It's difficult without the actual lock.

In the meantime, if anyone is interested I found that the SDK releases on gradle have the source code: https://repo.gradle.org/gradle/repo/com/tongtonglock/ttlock/. This should speed up things.

But isn't the data part encoded/encrypted? Could you give me a more practical example?

It should be encrypted, but not with those 2 basic functions. I will take a look at the code again to try see what the flow is to get those AES keys from the lock.

Fusseldieb commented 3 years ago

I'm still struggling with the BLE code that gives different results on scans on my mac and linux, still learning how it all works.

Yep, I also "learned" BLE just because of this lock.

It's difficult without the actual lock.

I can totally get that!

In the meantime, if anyone is interested I found that the SDK releases on gradle have the source code: https://repo.gradle.org/gradle/repo/com/tongtonglock/ttlock/. This should speed up things.

Awesome find! I will keep a copy of it locally in case of whatever. Also, I'll take a look again, without the mangled function/variable names it's probably easier (I hope).

It should be encrypted, but not with those 2 basic functions.

Didn't quite get this part. Which functions do you mean?

I will take a look at the code again to try see what the flow is to get those AES keys from the lock.

If you somehow get how they're encoded/encrypted, let me know and I'll try to decode/decrypt them with your suggested method and let you know!

kind3r commented 3 years ago

Didn't quite get this part. Which functions do you mean?

The encodeWithEncrypt/decodeWithEncrypt ones from the .so.

From the looks of it the only time that the SDK requests the AES keys from the lock is during the initialisation of the lock. After that it generates and admin password and unlock key and sends it encrypted to the lock. All this data (keys, passwords etc.) is sent back to the user in a json serialized LockData object via the InitLockCallback::onInitLockSuccess. It is then the responsability of the user to store this data so it can connect back to the lock.

Data sent to the lock is encrypted with AES, see AESUtil::aesEncrypt and aesDecrypt.

Fusseldieb commented 3 years ago

Nice!

the only time that the SDK requests the AES keys from the lock is during the initialisation of the lock.

In this case when it's totally reset and first-time paired? Did you find out how it's sent (in which format)? It's AES encrypted JSON? Are they individual bytes each meaning something different?

Data sent to the lock is encrypted with AES, see AESUtil::aesEncrypt and aesDecrypt.

Nice! Will take a look at that.

Is it AES already before pairing or only after, do you know that by any chance?

kind3r commented 3 years ago

I put together a small scanning tool in a private repository (I sent you an invite). You can follow the prerequisites from the noble repo, but I assume you already have those installed. Use

npm install
npm start

to get it running.

If it works at all, it should scan for locks and show some information. Scanning is filtered to the UUID from the original SDK (00001910-0000-1000-8000-00805f9b34fb). You can also change demo.js and pass an empty array as uuids parameter of the TTLockClient constructor, this way it will show all devices found.

I am curious if it does find a lock and if the data about it is retrived correctly. For me, for example, it does not find my 2 BLE temperature and humidity sensors, no matter what I tried.

square-spade commented 3 years ago

I put together a small scanning tool in a private repository (I sent you an invite). You can follow the prerequisites from the noble repo, but I assume you already have those installed.

OH! You mind spreading the love? Just got the lock yesterday!

I was tempted to try serial (my locks board is easy to get to and has enough space to put a piZero) and do it all direct. The BLE chip isnt the greatest.

kind3r commented 3 years ago

Hi @square-spade , I sent you an invite to the private repo. At the moment it does not do much, it should detect the lock if it is using TTLock protocols, and print out some information.

I received my lock this week but unfortunately it's a Tuya WiFi lock so tottaly different. Plus it does not fit my door and I have to make some adjustments (if at all possible).

I will be back on the project with new information soon, but keep in mind that this will be more of a proof of concept/library in the end (if it will work at all). There will be a lot of work ahead to integrate this into something usable in automations.

square-spade commented 3 years ago

Many thanks! I'll let you know what I get when I have a look!

unfortunately it's a Tuya WiFi lock so tottaly different.

Theres one on sale here for around $80 I might even get one to have a look, the new Tuya Dev platform is actually quite good. Well for removing Tuya completely obviously haha.

Plus it does not fit my door and I have to make some adjustments (if at all possible).

I'd strongly advise not to reposition, drill more, have to buy a brand new door... Can happen :/

kind3r commented 3 years ago

Theres one on sale here for around $80 I might even get one to have a look, the new Tuya Dev platform is actually quite good. Well for removing Tuya completely obviously haha.

My advice, don't, I have studied it for the past couple of days, the problems are like so:

I'd strongly advise not to reposition, drill more, have to buy a brand new door... Can happen :/

There are 2 issues I'm facing:

Not sure sild is the right word in english, I'm referring to the face plates that contain the door handle on the inside and outside of the door.

mnayef95 commented 3 years ago

I have checked the ttlock SDK and the demo application.

They are connected to Bluetooth using something called lock data which got it from the /v3/key/list endpoint, after that they just decode it using the default base64 decoder to extract a lot of information.

uid
lockName
lockMac
userType
lockVersion
adminPwd
lockKey
lockFlagPos
noKeyPwd
deletePwd
pwdInfo
timestamp
aesKeyStr
specialValue
featureValue
startDate
endDate
electricQuantity
timezoneRawOffset
modelNum
hardwareRevision
firmwareRevision
nbNodeId
nbOperator
nbCardNumber
nbRssi
version
factoryDate
ref
autoLockTime
lightingTime
resetButton
tamperAlert
privacyLock
displayPasscode
lockSound

This is the method they are using to decode it

   public static LockData parseLockData(String lockData) {
        LockData lockDataObj = null;
        try {
            byte[] decodeBytes = Base64.decode(lockData, Base64.DEFAULT);
            byte[] macBytes = Arrays.copyOfRange(decodeBytes, decodeBytes.length - 6, decodeBytes.length);
            decodeBytes = Arrays.copyOf(decodeBytes, decodeBytes.length - 6);
            String aesKey = DigitUtil.getMacByByte(macBytes);
            aesKey = aesKey.substring(0, 9) + aesKey.substring(10);
            decodeBytes = AESUtil.aesDecrypt(decodeBytes, aesKey.getBytes());
            if (decodeBytes == null) {
                return null;
            }
            lockData = new String(decodeBytes);
            lockDataObj = GsonUtil.toObject(lockData, LockData.class);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return lockDataObj;
    }

All utils used in this method can be found here.

And this is the data they are used to extract the information

Se4uPZszYojdOPKROyO0DXaU6OHOrsBWzaC2n8GYarBh7+AzXdCnEqhLk/EIAtkHIttt+VTs8NuksijGQFmi2C45Oh/znuRUkG5jXqjWWBXy/pkjsl/rrTs9hzu2lXUMWU8snx/bztvrO258n3lsCo4TgYVhFTyytmJmM0ldDv3RbCMIeJvm3UYtugm10lhtRtdFRwY8vhGg+PY7inM5mptOa2cMTz6Bwl0VF7VxQrC4mxxzJoiWZJ4YbLZZIw+9Wb7VqNnzdBGc88I9NmIXPkb8kVs4ryMgvrXTvq41R/d2upoA8GS8LdHOS5a+NYq2QUK1kJlCtFvVedwCm+eJtXd/M1tHaMNL6FObFZDRXekEitJhlT4xhGBar09QroxmT/KK/hsL5K+owPKhABP1TivTQVF8/XupF4ZJLtFvMDxi/TQaZQxjAZ7386XcAn2q35pSE2G1PEPnYe2pMEC0LyvQFBKsd0SCHi3bzhj6P1AaWM++5OedS2sLfDyF5vSs9VMP0emztMa/y7pbLFdfW0WpIgovtRBo0zZOsdXPvlmteFHflEzmrLZPePF/WnwCjg6aKZ94ykr6f9icCP1BIG6tVjk9F9TY5I2Njnck0OwqsRzTL7jjGT3GRtA+qem62NAchSkCPTHCyNK7rGWAWUWV3+lC0AT3llLzIcx/U9y1L05T84cZLcdq6OSC2qAgJziZMurNYqoZV8d+Ook4kJbIJrAlT8yCGDjLL9KTvh7Yqgj2DMrgG7m6PizOvmhCwIevgyRBvWK2NqGFBP2RutzuUg1aKQ==

That's what I got for now, is that will help to know how to control the lock locally?

kind3r commented 3 years ago

Hi @mnayef95,

At the moment we are not studying the TTLock cloud api, which is actually documented at https://open.ttlock.com/doc/api.

We are looking into the BLE protocol so we can make a lib that can talk to the lock without using the TTLock app/SDK etc.

The current status is we can scan for locks and read some information about them. Next step we are currently working on is initialising the lock (getting the AES key from the lock and adding initial administrator).

burner- commented 3 years ago

So you managed to get TTLock for your hands? Can I help somehow or sponsor your project? My final goal is get esp32 mqtt-bt bridge what allows me to lock/unlock and maybe even set pins without ttlock own cloud. I currently have 4 locks installed and plan to install more of those if I manage to avoid use of that buggy, and slow chinese cloud api/gw.

Infiniski commented 3 years ago

I am dieing to get my lock to remotely unlock with the help of an esp myself. Has there been any success?

kind3r commented 3 years ago

Hi, yes, there has been quite some progress.

There are still some bugs mostly due to the poor BLE signal of the lock and some missing features (most important to be able to receive notifications when the lock is manually unlocked or locked using the keypad, fingerprint card etc.) but it is usable.

Infiniski commented 3 years ago

That's excellent work, I only have the normal esp 32 does it still only work on the rovers? Also have a pi zero to hand could that work as a gateway? Bit of a beginner to all this apologies. If I get this working I'm buying you a couple of pints Thank you

square-spade commented 3 years ago

You could always modify the sdk-js to run the bear minimums on the pi and a simple NGINX server front-end to call the lock/unlock scripts (I had this setup when I was running garagePi, even had IFTTT integrated)... Granted the SDK runs on ARMv6! The potential security risks are a bit high though. You'd need to put to much effort in to secure the setup if you are exposing the server WAN side. Are you not running/planning on running any smart home platform at all?

Then again you could just buy the official gateway in the interim while you play around with the code (only around AUD$24 from AliExpress)

kind3r commented 3 years ago

I'm not exactly sure what you want to achive. The whole ideea of this projects was to integrate the lock into Home Assistant (HA) without using the TTLock cloud so it can work offline (plus the added benefit of not having others spying on your habbits), since most people use HA to control their smart homes. There are of course other home automation software out there (like OpenHab) which I don't have any plans to integrate into, but the SDK could be used as a base to create addons for those as well.

The esp software is designed to act as a proxy/gateway/range extender. SInce most people run HA on a PI (3 or 4) and the BLE (and WiFi) range on those devices is pretty limited, and also the lock's BLE being enclosed in metal is also poor, you can configure the SDK to use the BLE on the esp device that you can place closer to the lock. But the esp only works together with the SDK (or the HA addon that is built on top of the SDK).

You could install the SDK on the PI Zero and use the command line to first pair the lock and then lock/unlock, add fingerprints etc. But there is no UI and if you want to remote control it you'd have to figure out your own way of doing this.

Infiniski commented 3 years ago

Sorry I should have included more detail.

I have the ttlock integration installed on my pi, I haven't had the chance to check to see if it can pair yet but I do know it's more than five metres away from my lock, so I'm guessing I'm going to have to extend the range.

After reading your installation it said that it requires the esp rover board, but I only have the generic esp32 board. I noticed though you can modify the SDK to use it with a different board though so I'll try that. Thanks for the help!

kind3r commented 3 years ago

You could try first without esp, maybe you're lucky. I have only concrete walls in my appartment and depending on the position of the lock and the PI sometimes it works over longer distances, sometimes it does not. Guess my 3 WiFi APs and all the 30+ zigbee devices don't help either since the all use more or less the same frequency.

As for the gateway, you just need to add a new entry for the esp board you have in platformio.ini and it should work. I think it requires a bit over 300K of RAM at the moment, but this is only a requirement when you use the web config as each https connection requires about 50K of free RAM to be available. This is one of the reasons I'm trying to rewrite it in pure ESP-IDF as the Arduino components being very generic and beginner friendly use a lot of ram and program memory for a lot of features that are not required in this case.

burner- commented 3 years ago

Not so big fan of esp-idf. Why you need https support at first place? I think more usefull is just make simple mqtt - ble/ttlock bridge.

kind3r commented 3 years ago

TL/DR for a minimal security.

While initial configuration is done via AP mode which requires a password that secures the wireless communication (unlike open APs that are sniffable), after the gateway joins your WiFi anyone in your LAN could intercept that traffic and take control of the device, and worse, your other BLE devices (as my goal is to use this as a generic gateway).

MQTT is also plain text and by default unsecured so most people use it as such. Even if MQTT was secured, it would require the gateway to be configured to connect to the MQTT server, plus accepting self signed certificates, adding more complexity to the configuration options. It would still require another secure way for specifying those configuration options. Plus, it would add another actor in the middle instead of having a direct connection.

I know most people don't think about the security of their devices, especially if those devices are inside their LAN and 'not directly exposed' to the internet. They just want them to work and they are happy when they do, not thinking about the possibility of one of those devices being compromised and providing an attacker access to the rest of his private LAN and his other devices. And theese days attackers focus more and more on those types of small networked home automation devices with bad written code to provide them with access to networks that they can use to gather personal data or launch other attacks.

And we are talking about a lock, which is supposed to keep your appartment/house safe from intruders, so security is important IMHO, even if not bulletproof at least enough to discourage most attempts.

ESP-IDF is still the base for the Arduino port, indeed the lower level concepts are harder to grasp for new coders with simple projects, this is the whole reason for the Arduino's framework existance. But if you want to utilize the full potential of the esp32, like this project requires, you'd have to optimize your resources and only use exactly what you need, which is not possible when using the Arduino framework. At the moment it works, but it's at the very limit and it is very possible that it will crash after some time. And while the esp32 will restart and communication will resume, I would not call it stable.

In the future, there could also be a possibility to port the ttlock-sdk to the esp32, tho I'm not sure if there would be any real use case for it. It would still require some sort of UI and/or integration with home automation software to be useful and it would dedicate the esp to this task. Meanwhile, I'm trying to achive generic BLE gateway functionality that could be used in other projects as well. Think of it like a remote bluetooth adapter.

tahamv commented 2 weeks ago

Hi Guys,

Have you been able to identify the exact algorithm used in the libLockCore.so library for decoding the adminPs and unlockKey in LockData? @kind3r @Fusseldieb

Fusseldieb commented 2 weeks ago

Hi @tahamv . Yes, kind3r managed to do it. It works!

https://github.com/kind3r/hass-addons https://github.com/kind3r/ttlock-sdk-js

tahamv commented 2 weeks ago

There is a function in ttlock-sdk-js named decodeWithEncrypt, but it doesn't work correctly. @Fusseldieb

Fusseldieb commented 2 weeks ago

@tahamv Use the hass-addons repo, as I think it's more up-to-date than the ttlock-sdk-js one.