atc1441 / ATC_MiThermometer

Custom firmware for the Xiaomi Thermometer LYWSD03MMC and Telink Flasher via USB to Serial converter
2.77k stars 466 forks source link

Allow dumping original firmware via ATCtelink.py #14

Open znanev opened 4 years ago

znanev commented 4 years ago

Hi @atc1441,

First I'd like to thank you for the impressive work that you've done reverse engineering the flashing protocol and giving us mortals the possibility to flash custom firmware via OTA :)

I own some of the other Mi BLE sensors - LYWSD02, CGG1 and MHO-C401. Inspired by your work, yesterday I opened the latest sensor in my collection - MHO-C401 (greyish square body with e-ink display) , and I just confirmed my suspicions that its built around TLSR8251 too:

IMG_20200907_173110

So, I'd like to start experimenting with it, but not having the hardware flashing tool, I'd have to rely on USB to UART flashing and OTA. But first I'd like to be able to dump its original firmware, so that I have a predictable starting point should anything breaks.

Is it possible to make the ATCtelink.py script dump the firmware of the attached device into a file?

atc1441 commented 4 years ago

Good to see there is another one with the TLSR8251 the Mosquito repellent has an TLSR8261 so its a bit different.

The .py script is unfortunately only one way so only data can be pushed from the PC to the MCU but not in the other direction.

However it is possible to use this https://github.com/pvvx/TlsrTools or this tool https://github.com/pvvx/TlsrComProg to dump the flash of an TLSR8266 that one does not have the same SWS protocol so one aditional byte needs to be send like in my script.

Ordered one of the MHO-C401 to get into it when its here but it takes a while

It should also be possible to intercept the web requests when its asking for a new firmware and get the flash that way. can you please send me what data is in the BLE characteristics maybe its possible to emulate the MHO-C401 and i can get the update file for you

znanev commented 4 years ago

Thanks for this info, will look into the links that you provided!

Following are screenshots of all characteristics and their values of MHO-C401 sensor:

image image image

I couldn't find a way to get this information as plain text, hope the screenshots are fine?

atc1441 commented 4 years ago

Thank you that does help, can you maybe as well send me one of the Advertising data packets?

The app does recognize the Model that way.

But as your firmware is 1.0.0 its very unlikely that there is one on Xiaomi servers :-( when i receive mine will definitely dump the firmware

znanev commented 4 years ago

Yeah, being a new device, that was exactly my thought - that there won't be a new firmware on Xiaomi's servers.

Anyway, here is a raw dump of two advertisements (some 15-20 minutes apart):

0x0201060F1695FE305887038937445338C1A40809094D484F2D43343031
0x0201060F1695FE305887039337445338C1A40809094D484F2D43343031

In fact this new device is quite similar (regarding BLE) to LYWSD03MMC - I added support for MHO-C401 to mitemp_bt custom component for Home Assistant a couple of weeks back, you can see details about the device ID and capabilities here.

atc1441 commented 4 years ago

Thank you, with the help of that i was able to Emulate that thermometer and get the right Model name ZenMeasure

Model is: miaomiaoce.sensor_ht.h1

but unfortunately there is no update file available as i already guessed :-/ so waiting for the hardware is needed :D

znanev commented 4 years ago

Thanks for trying that out, though!

I'll wait for the firmware dump before trying to upload anything to my device. But my curiosity gets bigger with the hour, so I might ping you again when I start playing with TlsrTools :)

vevsvevs commented 4 years ago

@znanev here you are. b0e28c6e5932bba59fb19290e626b138_upd_miaomiaoce.sensor_ht.h1.bin.zip

atc1441 commented 4 years ago

Oh great. Do you own one ? @vevsvevs

atc1441 commented 4 years ago

Looks like the server only gives back update files if you fully added one to your account

atc1441 commented 4 years ago

Flashed that file to one of the LYWSD03MMC and it does fully work except the LCD of course. so temperature and humidity is showing in the mi home app, and activation worked as expected. So that means the I2C pins are on the same as the LYWSD03MMC.

@znanev can you maybe make some more picture including the back to trace some routes a bit more

Thank you @vevsvevs that way its already possible to reverse it a bit

atc1441 commented 4 years ago

Activation and flashing did directly worked with the Web Flasher, so in theory its no problem to flash a custom firmware, if one exists.

will do a logic analyzer run to get infos about the display controlling

it still not showing an update file even with the fully registered device, wondering where the file comes from :)

znanev commented 4 years ago

@znanev here you are. b0e28c6e5932bba59fb19290e626b138_upd_miaomiaoce.sensor_ht.h1.bin.zip

Wow, thanks a ton, @vevsvevs !

@atc1441 Here's the back of the PCB:

IMG_20200907_173427

I might try to flash it tomorrow with the firmware that @vevsvevs provided. I have another MHO-C401 ordered, so could sacrifice one for... research purposes ;)

znanev commented 4 years ago

Just chiming back in to report, that I could't wait till tomorrow.. so I went ahead and flashed the custom firmware (ATC_Thermometer.bin). As expected, everything BLE-wise was working fine - I was able to read all characteristics, change values etc. The only thing not working was the display, as expected. It might be possible that the e-ink display uses SPI, or some other protocol, or just has another address if it uses I2C. I'll probably have to hold checking this out till the weekend.

Flashing back the stock firmware (thanks again, @vevsvevs !) brought the display back to life... so, I'm more than happy.

Thanks again for the wonderful work, I'll definitely dive into this deeper :)

Edit: just to confirm another observation (I saw it mentioned somewhere in this repo) - the custom firmware indeed reports the relative humidity 5% lower than the stock firmware. Is this calibration constant "hard", set at 5%, or is it somehow dependent on something else? Just curious ...

atc1441 commented 4 years ago

Great to hear that.

The E-Ink display will be connected via spi plus busy, bs, reset and dc line. Most likely

The sensor data read in the custom firmware is simply what is comming from the sensor, so something is done by it on the stock firmware as its 5% different. I added the offset setting in the custom firmware if someone wants to make it these 5% off again.

atc1441 commented 4 years ago

Was able to find a software SPI function in the Firmware, and the pins also fit with the Chip pinout, so PB7 = SPI Mosi PD7 = SPI Clk

soft_spi

Also some init e-ink function is there, so most likely we will see these values again:

The first parameter of SPI pre seems to be for switching between data and command mode, but it does it via a pre clock pulse, dont know if that is correct

initEink

znanev commented 4 years ago

Whoa, impressive detective work, @atc1441 !

How did you get those display functions from the dump (i.e. what is the disassembler used) ? They look too pretty, with nice symbol names :)

atc1441 commented 4 years ago

That is with Ghidra, but renamed the variables/functions of course, its without any debug names on stock. But looking at what registers it access it possible to get the functions reversed, also by comparing self compiled code its more easy to reverse. Using this modul https://github.com/rgov/Ghidra_TELink_TC32

znanev commented 3 years ago

@atc1441 You are waaay ahead of me! Tried the Telink-TC32 module for Ghidra, but didn't get any meaningful output, so I'm opting out from reverse engineering the original firmware :)

I hope I'll find some time during the weekend to hook some probes to the e-ink display to see what can I get. Sniffing the SPI bus with BusPirate is on my tasks list too :)

atc1441 commented 3 years ago

Took a feew days to get into the TC32 and Ghidra

Good luck with sniffing the data :) waiting for any infos

znanev commented 3 years ago

Following some poking with the logic analyser during the weekend, here are the conclusions about the display in MHO-C401:

  1. The display is the same as the one used in the "dumber" version of this device - MHO-C201 (it just shows the temperature and humidity and doesn't have any radio built in). By display I mean the segmented EPD itself, without the driving IC.

  2. The EPD driver ICs in MHO-C401 and MHO-C201 are however different - so are the pin assignments on the 10-pin flat flex connectors:

MHO-C201 Pin # Function
1 VDL
2 VDH
3 GND
4 Vin
5 SHD_N
6 RST_N
7 SDA
8 SCL
9 CSB
10 BUSY_N
MHO-C401 Pin # Function Connected to
1 VDL C3 to GND
2 VDH C2 to GND
3 GND Ground
4 Vin +3V
5 SDA SPI_DO
6 CSB SPI_CK
7 SCL SPI_CS
8 ? PC4
9 ? PA6
10 BUSY_N PA5
  1. The protocol used to communicate with the EPD is 3-wire SPI (SPI enable - CSB, data - SDA and clock - SCL). Commands and data are denoted not by a dedicated wire (D/C as in other EPDs), but instead as first bit in the 9 bits sequence. Value of 0 denotes that following 8 bits are command byte. Value of 1 - following 8 bits are data byte. So SPI settings are as follows: Mode 0, 9-bits, MSB first. So this confirms @atc1441's suspicion above and matches the disassembly of function void init_E-Ink.

  2. The SPI commands and LUT data are quite different from those used in MHO-C201. I have 3 good update captures so far and intend to do much more, especially start up and full refresh.

  3. Found some really nice reverse-engineering resources for MHO-C201:

    Reverse Engineering a MHO-C201 Mostly c++ code examples

    Those gave me a good starting point and inspiration to reverse-engineer the MHO-C401 display protocol. So all credits about MHO-C201 go to those two resources above.

  4. I'll upload the SPI captures later. From the 3 updates I can clearly see a pattern of LUT tables load and data load, so knowing the exact commands meaning won't be necessary as long as it is clear how the segments are mapped in the data bytes. By the way those captures match exactly the code of init_E-Ink above. Most of the commands are the same in different EPD driver ICs implementations (Power on/off sequences, some LUT loads etc.), some are quite unique to MHO-C401's driver IC (0x18 seems to be the data load command) etc.

Hope my observations so far will be a good starting point in developing an EPD driver for MHO-C401 :)

Edited to add: Link to the TLSR8251 datasheet - the IC used in MHO-C401 is the 24-pin variant:

http://wiki.telink-semi.cn/doc/ds/DS_TLSR8251-E_Datasheet%20for%20Telink%20BLE+IEEE802.15.4%20Multi-Standard%20Wireless%20SoC%20TLSR8251.pdf

atc1441 commented 3 years ago

These are some very nice infos. cool.

I didn't even knew the C201 does not have Bluetooth, good i didn't bought one

Still waiting for mine

EDIT:

As the 9th bit is indeed for D/C here is also the spi_send_pre function from the stock firmware.

spi_send_pre

znanev commented 3 years ago

Nice disassembly of send_spi_pre !

Here I'm uploading the 3 partial display updates captures. Excel came quite handy for the initial analysis - at lest it's easy to see with a quick glance what changes and what not :)

MHO-C401-SPI-PartialUpdates-Captures.xlsx

Looking at the init_E-Ink function again, the captures confirm what is sent over SPI! I'll try capturing full display update and start up sequences later.

atc1441 commented 3 years ago

Just found more info on the extra bit for D/C

It looks like the BS pin on that Display is pulled high so the SPI is in 3-wire SPI mode.

that means the 9th bit is just simply DC but in the SPI data.

see here at page 30 https://www.waveshare.com/w/upload/6/6a/4.2inch-e-paper-specification.pdf so it seems to be still just normal EPD in general

The info about that is also very usefull for a different project right now so i am quite happy

znanev commented 3 years ago

Indeed this is the case, I confirmed it with the logic analyser, it's also visible in the Excel spreadsheet above - SPI is 9-bit ;)

I couldn't identify the manufacturer of the display or its particular driver IC however. Traversing the datasheets of different IC manufacturers, I was able to confirm that most of the "backbone" commands are the same - like 0x00, 0x02, 0x04 etc.). Also I suspect that commands 0x20, 0x23, 0x24, 0x25 and 0x26 just load static tables for the waveforms. It will be command 0x18 that will need more attention in order to decipher the segments mapping:

image

(picture is not mine - it comes from this repo.

atc1441 commented 3 years ago

Ah yes sorry you did wrote it exactly that way, sorry for misunderstanding, just saw it because of the other project and that reminded me of the 9th bit here.

Reversing a 4.2" E-Paper Pricetag display a bit more right now

znanev commented 3 years ago

Ah, no probs, it was just a heads up so you don't do the same again and lose time in the process ;)

apaperclip commented 3 years ago

@znanev replying here since your original comment mentioned other sensors. There is also the LYWSDCGQ, do you know if it contains the the TLSR8251? I did some searching and fcc id lookups but havent been able to find out.

There is a google sheet that maps the 4 sensors created by youtube user Chao-chao https://docs.google.com/spreadsheets/d/10qnYSn58TGHo94be__34PY7KUozqwq5Pr8S1KMtKvWY/edit#gid=934182368 I think it needs updated based on the work you guys are doing!

This has been great to follow!

atc1441 commented 3 years ago

As far i know that one uses an nRF51 SOC. So in generall its possible to write a custom firmware but very different from this here

znanev commented 3 years ago

@apaperclip I personally don't own LYWSDCGQ, but have its "successor" - CGG1 (round body with e-ink display). Haven't opened it yet, but its turn will come soon, for sure!

Meanwhile I have good news about MHO-C401's display - I was able to capture all necessary communication from the MCU and have setup a quick proof of concept project using Wemos Mini (ESP8266 based board). I managed to map all the segments, so will publish my finding soon. Stay tuned!

apaperclip commented 3 years ago

Thank you atc1441, i dont think it would even need a custom fw based on how well it seems to work and lack of encryption but wanted to check.

Great news znanev, I'm going to order something just trying to see what I should get. Things like lack of battery on screen or via ble seems like obvious gaps but then again, if it stops working - just replace the battery. I love the eink screen of the CGG1 but the reporting times are less than ideal based on that google doc.

apaperclip commented 3 years ago

i came across what appears to be some oem'd versions of two of the sensors. Perhaps you've seen them already. If not maybe there is some firmware, docs or fcc filings out there that could aid you.

This looks like the MHO-C401 to me https://www.amazon.com/Homidy-Hygrometer-Thermometer-Industrial-Temperature/dp/B07P6L3356/ref=sr_1_1?dchild=1&keywords=homidy&qid=1600542809&sr=8-1

This looks like the CGG1 https://www.amazon.com/Homidy-C9-Thermometer-Professional-Temperature/dp/B07WZNWWJK/ref=sr_1_19?dchild=1&keywords=humidity+bluetooth&qid=1600542959&sr=8-19

One thing that is interesting is that both can toggle C/F with a long press of the recessed button.

znanev commented 3 years ago

@apaperclip Thanks for those links, I haven't encountered those OEM versions, but by the looks of it the first one is not MHO-C401, but its "dumber" version - MHO-C201. I owned 3 of them and functionally are the same as MHO-C401, but lack any "smart" features like BLE. Indeed Celsius and Fahrenheit can be changed by that recessed button, but it's easier to just use the App to do this for the BLE-enabled devices.

The CGG1 device (manufatured by Qingping Technology (Beijing) Co., Ltd) seems to be registered with FCC:

https://fccid.io/2AQ3F-CGG1

I couldn't identify the ICs from the FCC photos, but if anyone is interested, I can make a teardown of my device.

atc1441 commented 3 years ago

@znanev the CGG1 SOC is definitely an nRF51 or nRF52 looking at the parts around it and the significant label on it

so custom firmware is possible, just needs to be developed^^

Also it has an external SPI flash, but that not important.

znanev commented 3 years ago

A short teaser ;)

MHO-C401-epd-screen-demo

Display is attached to an ESP8266-based board (WeMos D1 mini), running adapted version of this code. I will publish it once I tidy it up a little, as it's all a mess now :)

atc1441 commented 3 years ago

Perfect!

sermayoral commented 3 years ago

@znanev the CGG1 SOC is definitely an nRF51 or nRF52 looking at the parts around it and the significant label on it

so custom firmware is possible, just needs to be developed^^

Also it has an external SPI flash, but that not important.

The CGG1 model has a horrible refresh time, so it would be great if it would have this custom firmware.

Thanks!

atc1441 commented 3 years ago

It is more than likely that they save battery by refresh not so periodigly.

I havent ordered a CGG1 for now. Will think about doing so.

Currently i am waiting for the C401 and the Mosquito reppelent to arrive to hack

atc1441 commented 3 years ago

Git my C401 today 👍

Unfortunately i will not have time in the next days to "hack"

But the display looks really nice

znanev commented 3 years ago

@atc1441 Good, good!

I'll have to wrap what I've written so far and push it.. so you have a head start. Maybe I'll just leave the detailed process description for later..

znanev commented 3 years ago

I've just pushed my findings and code to https://github.com/znanev/MHO-C401.

There are some final touches missing (segments diagram, PCB photos and data captures), but I'll update the repo in due course.

atc1441 commented 3 years ago

Very nice library !

znanev commented 3 years ago

Thanks!

Credits for the library go to https://github.com/GitJer/XiaomiMiaoMiaoCe.

That's the repo that I forked and adapted for the display of MHO-C401 :)

Hope you'll find it useful when you find some time to play with your C401 device ;)

znanev commented 3 years ago

@atc1441 Just pushed the missing files, reformatted the Readme and added more details about the reverse-engineering of the MHO-C401's display: https://github.com/znanev/MHO-C401.

Give me a shout if/when you need more information about MHO-C401's display, I'd be happy to help :)

michapr commented 3 years ago

Hi, @atc1441 is there any progress or work on a custom firmware for MHO-C401? It would be really great! I have some such sensors - and all of them have empty battery after 2-3 months about - if make a request every 8 minutes. Get the data from advertising information (like for LYWSD03MMC ) would be really nice... I know about the e-ink "problem", that's why asking ;)

Thank you for your work and time!

znanev commented 3 years ago

Hi @michapr ,

you could use the current firmware as is, if you don't need your sensors to visualise the readings (i.e. they are in attic, cupboard etc.). The BLE part is working fine and you can use the advertising data, without the need to connect to the sensors and thus drain the battery.

atc1441 commented 3 years ago

You can also use the activation method to get a sign key so you can use the stock firmware with advertising for now.

I havent worked on the custom firmware till now

michapr commented 3 years ago

You can also use the activation method to get a sign key so you can use the stock firmware with advertising for now.

But they are only sending the temp data once every 10-15 minutes, right? If no missing any advertising... I wanted to use it in HA, in other cases I get the data from other sensors (Z-Wave, 433MHz, BLE) very often. That's why was thinking about new firmware.

Is it not the same case as for LYWSD03MMC ? It was working also with sign key via advertising, right? Or is here any difference I do not know?

if you don't need your sensors to visualise the readings

Sensors are room sensors - so would be nice to see the temp. That's why I choose this sensor, because of the good display. LYWSD03MMC you cannot read in some cases.

Thanks!

znanev commented 3 years ago

@michapr Check this out, regarding HA:

https://github.com/custom-components/sensor.mitemp_bt

I use this custom component in order to passively listen to advertisements sent by LYWSD02, MHO-C303 and MHO-C401. For MHO-C401 you'll just have to configure the encryptor key (Mi Bind Key). Indeed MHO-C401 advertises once every 10 minutes, but that's why there are efforts here to provide alternative firmware for it.

michapr commented 3 years ago

@znanev I know this github projekt. I'm using OpenMQTTGateway and RPIEasy - that's why the integration of advertising data would be more simple here, even for other users ;)

Thank you, will wait, maybe the alternative firmware will be available in future ;)

pvvx commented 3 years ago

The .py script is unfortunately only one way so only data can be pushed from the PC to the MCU but not in the other direction. ComSwireReader825x.py https://github.com/pvvx/TlsrComSwireWriter