search

banksy-git / lidl-gateway-freedom

Freeing the Silvercrest (Lidl/Tuya) Smart Home Gateway from the cloud.
https://paulbanks.org/projects/lidl-zigbee/
GNU General Public License v3.0
206 stars 66 forks source link

Step by Step guide pls #14

Open fahadshery opened 3 years ago

fahadshery commented 3 years ago

Hi Paul,

Amazing project! I already have a lidl silver crest gateway up and running which I wish to integrate with Home Assistant.

Would appreciate a step by step n00b proof guide. Thanks

Cheers,

fahadshery commented 3 years ago

which 3.3V TTL serial port device can be used to connect to it? thanks

challs commented 3 years ago

which 3.3V TTL serial port device can be used to connect to it? thanks

I have one that looks like this, which is a FT232RL FTDI

challs commented 3 years ago

I already have a lidl silver crest gateway up and running which I wish to integrate with Home Assistant.

There is a lot of information over at the Home Assistant forum thread. I suggest you look through that. I wrote a small howto for the first stage here.

Note that if you have already connected your gateway to the internet and it has contacted Tuya's cloud, you will might not be able to use the root password extraction method, and you will need to rebuild and flash your root filesystem.

There is also further information in other issues such as here.

Back to your original reason for opening the issue: It probably needs someone to write it all up and propose an updated README. Maybe you have some time to help with that?

fahadshery commented 3 years ago

Hi @challs,

thanks for your detailed reply.

I already have UART to USB device which looks similar to yours. So I think it should be alright.

Note that if you have already connected your gateway to the internet and it has contacted Tuya's cloud, you will might not be able to use the root password extraction method, and you will need to rebuild and flash your root filesystem.

sadly, I have connected it to the internet prior to all this home automation interest. Now that I have installed HA and connected couple of sensors and CC2531 which I am in the process of replacing with CC2652R, I would love to use this gateway (Presumably, it has better speed and coverage).

Back to your original reason for opening the issue: It probably needs someone to write it all up and propose an updated README. Maybe you have some time to help with that?

I happily volunteer to write up a detailed n00b proof step by step instructions for you. Perhaps with a video tutorial. The first step is to open up the device. This is where I am stuck at the moment lol. I tried different tools etc. and I am unable to crack open the device. Any ideas?

MattWestb commented 3 years ago

The LIDL version is well snapped and is not so easy getting popping up. In the corners is one double snapping thug and also in the middle on each side so 8 that is holding the 2 parts together. You need some very thing metal or hard plastic and starting in one corner and snapping all thugs around 3 sides before its popping up.

fahadshery commented 3 years ago

I am making progress!

I have purchased UART to USB device, DuPONT cables and most importantly popped open the case. On Paul's website, I see there are pins sticking out of the Lidl PCB. But in my device there are none. I am assuming its some sort of pin header. Is that true? If it is what is it called?

Here is the picture PCBTop

thanks

MattWestb commented 3 years ago

The J1 is not populated from the factory but you can soldiering one if you like. Or you is using DuPont cable with pins and putting i holes and have little pressure so they is having good contact.

Pin 1 is on the inner side to the CPU and the pin 6 is on the side to the border of the PCB.

fahadshery commented 3 years ago

The J1 is not populated from the factory but you can soldiering one if you like. That's what I hate but might have to do it anyway!

Or you is using DuPont cable with pins and putting i holes and have little pressure so they is having good contact.

or I will give it a shot first.

Pin 1 is on the inner side to the CPU and the pin 6 is on the side to the border of the PCB.

I am not sure what you're trying to say here. I thought I just need to connect three cables i.e. TX, RX and GND?

Lastly, Which software do you use to connect to SERIAL port? I am using putty. I checked what the COM port is i.e. COM3, then set the speed to 38400 and connect.

is this the right method?

MattWestb commented 3 years ago

You can supplying the ZBGW with 3.3 V from the USB-TTL converter (plus the 3 normal GND, RX and TX) and no USB connected or you have the USB power supply connected and using GND, RX and TX. I was using the 3.3 V from the USB-TTL converter and not the USB power supply.

PuTTY is little old but its working OK as long not need sending files with XModem and in Linux Mini com i the standard.

MattWestb commented 3 years ago

If you have the right comport and the cable OK you shall see the logging from the system boot in the console then its booting and also if you is not having Ethernet connected its trying discovery the network and doing printing that you shall see. Then you can see the console long you can start trying booting in the boot loader but first you must having the comport working OK or its not working.

fahadshery commented 3 years ago

ok, I just saw the terminal messages for the first time and it lands me at a tuya login prompt looking for a username. is it normal?

MattWestb commented 3 years ago

Then you only need reboot the ZBGW and pressing "ESC" for landing in the bootloader :-)))

MattWestb commented 3 years ago

The password you must "extract" from the bootloader then you can logging in on the "normal" mode.

MattWestb commented 3 years ago

https://paulbanks.org/projects/lidl-zigbee/root.html

fahadshery commented 3 years ago

All done

Thanks very much.

I think I will change the README and do a PR.

Cheers

MattWestb commented 3 years ago

Great work done !!

I can recommending using the Release-1.2 of the binaries that have more functions and fixes that the linked from the main hacking page.

fahadshery commented 3 years ago

Great work done !!

I can recommending using the Release-1.2 of the binaries that have more functions and fixes that the linked from the main hacking page.

I downloaded everything using this page.

I have no idea which version I installed

fahadshery commented 3 years ago

ok I just checked the md5 hash for both binaries and they are the same :)

md5 serialgateway
MD5 (serialgateway) = 4bb59358e6db08192e8aeda3f8d0e646

md5 serialgateway.bin
MD5 (serialgateway.bin) = 4bb59358e6db08192e8aeda3f8d0e646
MattWestb commented 3 years ago

Its the original version that have problem doing updating of the Zigbee module. The newer is larger if you is looking on the file size.

I can also recommending updating the firmware on the Zigbee module but only if you is not planing to connecting the ZBGW to tuya cloud then its not (easy) to reverting the firmware update.

Edit: I was downloading the serialgateway for 2 weeks ago and its was the old version but i can being updated on the web site. Is the size the same on your hard disk if yes then its the same (and the MD5 is also the same).

Chris5020 commented 1 year ago

I always stuck on stopping the boot process into the bootloader. I'm connected to the gateway with putty and I see the messages. I tried normal ESC, also STRG+4, STRG+5 and STRG++, which I also found as escape chars, but nothing happens. Anyone a hint for me please?

regards, chris

MattWestb commented 1 year ago

Important is flow control is set to non (no hardware or software flow control) if i remember right. Also on Windows 10 with extra ptty the normal ESC is the right one for stopping the bootloader.

One warning is that tuya have released new versions with Matter support but i dont knowing if they have blocking our hacks or only updated there application they is running on the hardware. Also not all brands dont like paying for the upgrade so all ZBGW devices of different brands is not getting it but we dont knowing if / now and then if its happening.

Gianniva commented 1 year ago

Las year I have hacked 2 of the lidl zigbee gateway, went very smooth. I just bought a new gateway from lidl and now is impossible to stop loading linux. esc doesn't work anymore

banksy-git commented 1 year ago

Is it the same still as this device? https://www.lidl.it/p/smart-home/gateway-lidl-home/p60735 ; Maybe they changed the bootloader break sequence - or disabled it altogether. Are there any physical differences you can see (labels, etc)?

I'll check in Lidl next time I'm there and buy another. I'm sure we can find another way! :smiley:

MattWestb commented 1 year ago

LIDL have one new version certificated but is running the same version of the APP (1.0.6) and both is having hardware version 100 = not grantee that the bootloader is changed. They have getting one update to 1.0.9 09/08/2021.

The tuya version is original HW 1.0.2 and APP 1.1.14 and one HW 1.0.3 is running the same APP as 08/13/2021.

But is one updated HW V1.0.4 as 11/01/2022 that is running APP V2.0.0 and is Matter certificated and not Zigbee but i think its not being used of our rebranded devices then its very new.

I think they have updating the bootloader or the setting for it in the device so its not easy hacking it.

By the way great to see you Paul !!

Chris5020 commented 1 year ago

Gateway bought 2 weeks ago seems to work, I tried now another serial converter, and I was able to stop boot process. Now I'll try next steps PCB ist REV1.0.2.7 from 21 19

Chris5020 commented 1 year ago

OK, I followed the manual from here https://paulbanks.org/projects/lidl-zigbee/ha/ almost everything worked fine, but I had a bug transfering the serialgateway via ssh from windows, the destination file has wrong size and generates a segmentation fault. -rwxr-xr-x 1 root 0 69485 Dec 8 2022 serialgateway-defect -rwxr-xr-x 1 root 0 69197 Jan 1 00:19 serialgateway

now I was able to integrate it into my HomeAssistant.

One thing I'm not sure If the steps in the other howto with replacing the whole firmware is also required, or this is enough? https://paulbanks.org/projects/lidl-zigbee/

Thanks to all for your responses

MattWestb commented 1 year ago

Great work done !!

The original was running EZSP 6.5.X firmware on the Zigbee module and its little old and have many bugs that we dont like to have in our system. If you is adding it in ZHA you is getting the firmware version in the log so you can getting it easy.

If you like flashing the Zigbee module firmware in one easy way you can trying https://github.com/puddly/universal-silabs-flasher that is working good for my after updating my python install in windows.

Edit: If you must replacing the whole firmware ? The system firmware is not needed if you have getting root access and getting the serial server working then it was the first version patching the root password.

Gianniva commented 1 year ago

Hi, Ok I see that the problem can be the USB to TTL device, I have tried with 3 different usb to ttl and I cannot stop the boot loader. Here is the GW board and the 3 usd to ttl devices

Il giorno gio 8 dic 2022 alle ore 16:18 Chris5020 @.***> ha scritto:

Gateway bought 2 weeks ago seems to work, I tried now another serial converter, and I was able to stop boot process. Now I'll try next steps PCB ist REV1.0.2.7 from 21 19

— Reply to this email directly, view it on GitHub https://github.com/banksy-git/lidl-gateway-freedom/issues/14#issuecomment-1342807961, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHYUDHSE5LYHO5244RZLOFTWMHU45ANCNFSM46HYMZ2Q . You are receiving this because you commented.Message ID: @.***>

jf2020 commented 1 year ago

OK, I followed the manual from here https://paulbanks.org/projects/lidl-zigbee/ha/ almost everything worked fine, but I had a bug transfering the serialgateway via ssh from windows, the destination file has wrong size and generates a segmentation fault. -rwxr-xr-x 1 root 0 69485 Dec 8 2022 serialgateway-defect -rwxr-xr-x 1 root 0 69197 Jan 1 00:19 serialgateway

I had the same issue. Doing the transfer using WSL worked fine.

VGerris commented 1 minute ago

first of all, thanks Paul for a great write up, I was able to pull it off on a USB-c zigbee version of the gateway I bought not too long ago. Regarding noob guide, I was also missing simple things that are mentioned here, like pin 1, that RX and TX cross and more of the basic anyone doing it more often probably knows. I may do a write up myself sometime. Just to share, I have some V2 version of the board and I used minicom on linux to connect like so : sudo minicom -D /dev/ttyUSB0 using a cheap USB to serial device. thank you!