Flashing Tasmota device OTA with esp-homekit-devises

tonysprenk commented 4 years ago

Is it possible to flash a device that is running Tasmota with esp-HomeKit-devices OTA? When I try to upgrade the firmware with main.bin from HAA I get an error: “ Magic byte is not 0xE9”. Thanks for any help on this!

mafyata commented 4 years ago

Did you erased the old firmware first?

pawelosin commented 4 years ago

I did it using this https://github.com/ct-Open-Source/tuya-convert/issues/389#issuecomment-554106217

GPL71 commented 4 years ago

Search vglucas repo here in github ... you'll find an homekit flasher

peros550 commented 4 years ago

I did it using this ct-Open-Source/tuya-convert#389 (comment)

Would you please share the steps you followed to install HAA using the Tasmota firmware update method? After the installation you have OTA functionality through LCM?

RavenSystem commented 4 years ago

Tasmota is not Tuya, but you can try using firmware from "Tuya-Convert OTA Version": https://github.com/RavenSystem/esp-homekit-devices/wiki/Installation#third-party-installation-tools

After the installation you have OTA functionality through LCM?

Yes, you install LCM firmware and then you setup repo, wifi...

RavenSystem commented 4 years ago

And please, comment and results. ;)

peros550 commented 4 years ago

Sure! Stay tuned :)

peros550 commented 4 years ago

Good news! I have used the steps as described by @tonysprenk on the reddit page and LCM was installed on my Blitzwolf BW-SHP6.

Many thanks! Very helpful!

May I add a suggestion? When device is in Setup mode, a LED could be slowly blinking to show the status? It would be very helpful because LCM takes a few minutes to complete and when you have no serial connection to the device you are blind :)

pilot1981 commented 4 years ago

Can you explain here each step?

peros550 commented 4 years ago

Sure, just give me a few time to write a ~~mini~~ guide :)

peros550 commented 4 years ago

The process is very easy but if you haven't done so before it might feel complicated. You will need a raspberry with WLAN capability and a secondary device with wifi capabilities (i.e. mobile, tablet,pc). I had a rpi 2 with a usb wifi adapter, but it should work with rpi3/rpi4.

To my understanding, all devices running tuya software are looking for a known SSID network when are in pairing mode. Once they are connected to that network, they can be updated. This is exactly what the tuya-convert does. It creates an AP on the RPi and emulates the update process. It can serve Tasmota, ESPurna or any other software we like, in our case LCM which will lead us to HAA.

Note: this process may lead to a bricked/faulty device. It worked for me, but may not work for somebody else. I won't be able to provide help if something goes wrong and your device is bricked. If you want to take the risk though, then here are the steps I followed:

1) First of all I strongly suggest to prepare a SD card with the latest clean Raspbian OS and no other software on it. Raspberry should be connected via LAN cable to your network because its WIFI will be used for creating an AP. As soon you are able to log into the raspberry you should be good to proceed with the next commands.

2) run the following commands:

# git clone https://github.com/ct-Open-Source/tuya-convert
# cd tuya-convert
# ./install_prereq.sh

During the above steps, I was asked to deactivate some processes. I confirmed everything with "Y" as I didn't mind. You are suggested to do the same. That's why working on a brand new Raspbian installation matters.

3) Make sure your wifi adapter is identified as "wlan0" by running the following command # iwconfig

If by any chance your wifi adapter is identified as "wlan1" you will need to use your favorite editor and correct the following file. In that case, the command would be : nano config.txt

4) Download the file from this link and put it into folder ~/tuya-convert/files/

5) You are now able to start the AP process by giving the next command: ./start_flash.sh Wait until you reach to a point where it says "3. Press ENTER to continue"

6) Take your secondary device and connect to a wifi network called "vtrust-flash". I don't know why this is needed but I did it as this was part of the process. There will be no other use for the secondary device.

7) Take your smart IoT device and put it into pairing/setup mode. For me, all I needed to do was press the button for a few seconds. Then the led started blinking fast. Risk mode ON

8) Press Enter to the rpi terminal and watch the process. It will take a few seconds. Mainly, the process backup the current factory firmware and then loads a temporary firmware to the device that will help you load the third party firmware you want. I believe this is the point where no restart is allowed as it would negatively affect the whole process.

9) As soon as the previous process is completed, a few options are offered. Option '0' loads the original firmware back. Option '3' will load the thirdparty.bin (supposingly you did copy the file in step 4). Press 3 and wait. The process will take a few seconds and once done the IoT device will restart.

This was it. If you search for Wifi networks you will find LCM-xxxx, the rest of the process is well known.

Extra Notes to keep in mind: Please be extra careful in what you do. Since those IoT devices do not offer serial PINs like Sonoff devices, it will be very difficult to solder wires and flash using the normal method.

Although the tuya-convert process saves the original firmware, if you run all steps of the above guide, you will end up with LCM firmware hence you won't be able to go back to original firmware without soldering wires.

Good luck!

Rocket200 commented 4 years ago

Thanks to @peros550 @tonysprenk @RavenSystem @kueblc today I already flash 4x Gosund SP111-2 to Tasmota with tuya convert but later I read about esp-homekit-device and this is what I want.

but I want to know can I flash the esp-homekitFW with Tasmota ? or I need to use tuya convert again ? how can I bring the SP111 Plugs back I pairing mode to use tuya convert again but flash it with the thirdparty.bin?

Thanks for the Guide!

tonysprenk commented 4 years ago

As far as I’ve tried it’s not possible. There is a “upgrade” mechanism built into tasmota where you can upload a .bin file but it won’t take main.bin from HAA and whenever I’ve uploaded compiled HAA firmware (https://github.com/ct-Open-Source/tuya-convert/wiki/Flash-a-multipart-binary) it has bricked the device. So proceed with caution. Tuya-convert can only be used on a device running Tuya firmware. Just thinking of it now you might be able to upload the firmware used by ESPOTA (https://blog.stigok.com/2018/06/13/esp8266-ota-updates-command-line.html) through the tasmota upgrade mechanism after which you can use ESPOTA to upload esp-HomeKit-devices main.bin. Good luck!

Rocket200 commented 4 years ago

My English is Not Good and i can Not connect the SP111 with FTDI (Serial)

Is there no way to go back from Tasmota to other FWs? (without Serial)

Rocket200 commented 4 years ago

@tonysprenk i Need to Open the sp111 case and solder cable to some Points..? or can i make ESPOTA Same way as i did with tuya convert ?

kueblc commented 4 years ago

You would need an intermediate firmware in between Tasmota and ESPOTA as they use different bootloaders. The intermediate firmware we use in tuya-convert could be modified for this purpose. I may be able to help put that together.

peros550 commented 4 years ago

@kueblc hey this sound like very good news! If possible please give us some help on how to go from tasmota to LCM

I'm very keen to test the process and report back. I have a few spare sonoff basic running tasmota ready for test :)

kueblc commented 4 years ago

I had some time today so I took a closer look. I was mistaken earlier in saying that an intermediate firmware would be needed, turns out the eboot in Tasmota can be used to do the upgrade just fine.

Older versions of Tasmota (<7.1.2.6) should be able to upload the assembled multipart binary as is. Newer versions of Tasmota include a Tasmota version check. Assuming you want to upload a Tasmota firmware, this ensures they are compatible versions. If you try uploading a different firmware, you'll get a very unhelpful "incompatible" error message.

Luckily we can trick the version check pretty easily as it just looks for some magic bytes. All we need to do is append this marker with a high "version number" and it will be accepted by Tasmota. The magic sequence is A55AA55A FFFFFFFF 5AA55AA5.

I've gone ahead and made the above modifications to the multipart homekit binary, tested and working fine on my development board. Attached here if you would like to try it for yourself.

homekit-from-tasmota.zip

Rocket200 commented 4 years ago

and this i can Flash from every TasmotaFW to the homekitFW? or is this for tuya convert?

kueblc commented 4 years ago

You can install from Tasmota (or any other Arduino based ESP firmware) and also with tuya-convert.

Rocket200 commented 4 years ago

I try it from Tasmota 8.1.0 it but not working also with the tasmota-minimal.bin 8.1.0

Upload Failed Program flash size is larger than real flash size

I try 1 Gosund SP111 Plug to Downgrad from 8.1.0 to 7.2.0 all OK. then I try to go from 7.2.0 to 7.1.1 and then after the update the Plug not come back and stop working.

kueblc commented 4 years ago

I see that I forgot to change flash settings back from my development board. I fixed this and just updated the link, download the updated binary and try again.

Rocket200 commented 4 years ago

i Flash the next Gosund Sp111 from Tasmota to HAA with the new File (updated link) and i see now the AP LCM-XXXX but I not can connect to it.

maybe same problem here: https://github.com/HomeACcessoryKid/life-cycle-manager/issues/18 https://github.com/RavenSystem/esp-homekit-devices/issues/703

i solved the Problem the AP have IP 192.168.4.4 but I need to connect to 192.168.4.1 what OTA resorsitory need to use? now this stand inside: HomeACcessoryKid/ota-demo or change it to: RavenSystem/haa. ?

I can Not finished the AP Setup i always come back to the AP Settings (Waiting 10 Minute After Click Join Network)

tonysprenk commented 4 years ago

I’ve had this same problem with every device I flashed from tasmota even before @kueblc made the special firmware. Looks like the firmware is not written to the esp in the right way. See issue #650 . In that issue we came to the conclusion that the only option would be to open the device and flash it by serial so soldering wires to it to connect it to a computer.

kueblc commented 4 years ago

If you are connecting by wire and are able to retrieve a backup of the firmware, I can use this to further debug what went wrong. Can you clarify if this is an issue only when starting from Tasmota or does it occur when converting from Tuya firmware?

tonysprenk commented 4 years ago

I don’t have the ability to connect by wire, yet. If and when I do I will post a backup.

unscel commented 4 years ago

If you are connecting by wire and are able to retrieve a backup of the firmware, I can use this to further debug what went wrong. Can you clarify if this is an issue only when starting from Tasmota or does it occur when converting from Tuya firmware?

Hi kueblc, You can find backup of the firmware below. [https://1drv.ms/u/s!AiTGTuFwWDphk0qJ0zK8yA7sp05j?e=1cxuqq]

This (homekit-from-tasmota.zip) has been uploded to sonoff basic via firmvare verison 8.1.0.3 OTA upload firmware function. As mentioned above, I connected to device through 192.168.1.4 but "join" button does not seem to trigger any action and I could not proceed. Let me know if I can help further.

i3laze commented 4 years ago

Couldn't wait any longer, so here's my trial. I've flashed Sonoff T1 wall switch from Tasmota 8.1.0 to LCM/HAA using recent "homekit-from-tasmota.zip". Serial log including upgrade, LCM config via Serial and then WiFi Manager: https://pastebin.com/pX5h0rkG Backup of the flash running kueblc's custom LCM: https://yadi.sk/d/kNQU4B6wr1UnVw

Most interesting parts are:

WiFi setup indeed does nothing and goes round and round:

>>> wifi_config: wifi_config_server_on_settings_update: Update settings, body = ssid=Sweet+Home&password=SSID1PASS&otarepo=RavenSystem%2Fhaa&otafile=main.bin&otabeta=0
>>> wifi_config: wifi_config_station_connect: No configuration found
>>> wifi_config: Client disconnected

LCM has an option to input WiFi and repo settings directly into serial console. With this custom binary it leads to BSOD:
```
Enter the wifi password or <enter> to skip
Result:
Fatal exception (28): 
```
Just a reminder how a good LCM OTA looks like: https://pastebin.com/dR5NAXDk Specifically it doesn't have this at LCM boot: WARNING: Could not initialize sysparams (1)!

tonysprenk commented 4 years ago

@kueblc

kueblc commented 4 years ago

Thank you @unscel and @i3laze for collecting this data. I apologize for the late response, for we have had a recent loss in the family. I will try to get around to this later this week. Please feel free to ping me if I do not follow up soon.

i3laze commented 4 years ago

@kueblc, my deepest sympathies to you and your family.

tonysprenk commented 4 years ago

@kueblc my thoughts go out to you and your family!

unscel commented 4 years ago

I am sorry for your loss Colin.

kueblc commented 4 years ago

Thank you all.

I was able to work on this a bit today. Unfortunately, I had the same symptoms whether I used the posted binary or a clean install on a freshly wiped flash.

My hope was that the difference would become apparent when comparing a working backup with a corrupted one. Would someone be able to provide a backup of LCM that is configured and working as expected on your hardware?

unscel commented 4 years ago

Thank you all.

I was able to work on this a bit today. Unfortunately, I had the same symptoms whether I used the posted binary or a clean install on a freshly wiped flash.

My hope was that the difference would become apparent when comparing a working backup with a corrupted one. Would someone be able to provide a backup of LCM that is configured and working as expected on your hardware?

Hi @kueblc , There it is https://1drv.ms/u/s!AiTGTuFwWDphk0zZJ_l1wjmmsBej?e=6Dajxv

i3laze commented 4 years ago

Link above didn't work for me.. (actually does by copy&paste)

So.. here's same Sonoff T1 erased and flashed with just vanilla LCM (rboot+blank_config+otaboot): https://yadi.sk/d/gQ0C1wkBze1h4g

By the way, LCM development is in progress and there might be extra fixes useful for the task: https://github.com/HomeACcessoryKid/LCMdev/releases

tonysprenk commented 4 years ago

@kueblc any news on this?

peros550 commented 4 years ago

Was anybody successful on installing HAA on a device running tasmota using OTA?

i3laze commented 4 years ago

Was anybody successful on installing HAA on a device running tasmota using OTA?

@peros550, Last image released by @kueblc was missing some variables and didn’t boot past LCM. Since then a new OTA Boot image was released by RavenSystem. Hope will have this new one, when @kueblc returns back on track.

Crazor commented 4 years ago

So I also managed to "brick" a Gosund SP111 with the image @kueblc provided. The device boots with the LCM-* AP, I can set the OTA repo and Wifi credentials, but upon clicking Join, nothing happens. No reboot, no activity, nothing in the netcat log. I've successfully flashed a Teckin SP22 with TuyaConvert and a self-assembled image containing the three bin files. This same working file was not accepted by Tasmota OTA (claiming that it is too large for the flash after I appended the magic string). Although the file from @kueblc was larger than the one I cobbled together.

pashdown commented 4 years ago

Is there an image to get off of Tasmota that won't brick?

shanghaix commented 4 years ago

My sonoff R1 was running Tasmota and I was able to successfully flash HAA using the serial connection. (First erase flash then flashing the 3 files).

i3laze commented 4 years ago

My sonoff R1 was running Tasmota and I was able to successfully flash HAA using the serial connection. (First erase flash then flashing the 3 files).

Serial is old school! This thread is about migration using wireless OTA. No luck so far(

dopicchio commented 4 years ago

hi guys, i own a sonoff mini with tasmota and would like to switch to haa. is it possible via tuya converter? regards

MikeDProducer commented 4 years ago

Once you flash the device and connect to the WIFI that it creates, you enter in the IP to be able to configure your wifi settings, correct?

Is this where I can set a PIN for HomeKit?
If you ever need to change your home's wifi...does the device go back into pairing mode the same method used originally? For a smart bulb it is usually a series of turning the power on and off.

kueblc commented 4 years ago

@tonysprenk @i3laze et al

I've rebuilt an OTA upgrade image using the latest release from @RavenSystem using the process documented here. I've also appended the magic sequence as described here so you should be able to upload this to a device running Tasmota. YMMV.

Tested on a dev board, it seems to work fine, though the initialization took a while, ~10 minutes, and restarted multiple times. I haven't tested any function beyond setup as I do not have homekit.

Try at your own risk, homekit-from-tasmota.zip

i3laze commented 4 years ago

Greatly appreciated! :-)

Will give it a try later today.

Crazor commented 4 years ago

I get "Upload Failed" with the reason "Program flash size is larger than real flash size" while trying to flash this to a Tasmota 8.1.0.2 device, which is where I failed last time when I tried the steps @kueblc linked myself.

Rocket200 commented 4 years ago

i think kueblc forgot „to change flash settings back from the development board“ same bevor ...

maybe :)

i3laze commented 4 years ago

I’ve just upgraded Tasmota 6.6.0 pretty easy (few manual reboots might be needed). However I need a valid JSON to go beyond HAA Setup webpage, so I can’t confirm I saved a config (previous issue) and reached the final working state yet.

RavenSystem / esp-homekit-devices

Flashing Tasmota device OTA with esp-homekit-devises #683