search

ct-Open-Source / tuya-convert

A collection of scripts to flash Tuya IoT devices to alternative firmwares
MIT License
4.67k stars 501 forks source link

Native HomeKit Support #389

Closed tonysprenk closed 5 years ago

tonysprenk commented 5 years ago

Hi all!

Thanks for all the hard work that has been put into Tuya-Convert!

I have been successful in flashing curtains and wall sockets with Tasmota and espurna (wil add to the wiki) but my end goal is to have native HomeKit support.

This project makes native HomeKit on esp8266 chips possible but it doesn’t support OTA flashing.

OTA flashing is a prerequisite for my situation as I have a lot of built in wall sockets.

Is there anyone smarter than me that could point me in the right direction to getting Tuya-Convert to flash the necessary bin files to enable native HomeKit support?

Thanks!

kueblc commented 5 years ago

tuya-convert only works with Tuya firmware. Once the devices are converted to thirdparty firmware (Tasmota, Espurna, etc), tuya-convert can't do anything. To flash from Tasmota/Espurna to another firmware, you'll need to use their update mechanism.

If you are still running Tuya firmware, you may be able to use tuya-convert to flash any thirdparty firmware such as the one you linked, presuming it has met the requirements (must be under 512KB and include the full bootloader).

tonysprenk commented 5 years ago

That’s the thing, the esp HomeKit firmware is designed to be flashed on a completely empty chip so I guess it makes it impossible to be flashed OTA. Hopefully someone smarter than me can come up with a solution. I think the ability to be able to convert Tuya devices to be natively HomeKit compatible would be pretty neat!

kueblc commented 5 years ago

I'm not sure why it shouldn't be possible to flash OTA, unless it uses more than 512KB of flash. The Arduino ESP OTA mechanism will download the file, and use the bootloader to move it to the correct place on next boot, overwriting whatever is already there. tuya-convert takes care of cleaning up the system param area of flash so it will be more or less a clean slate.

tonysprenk commented 5 years ago

Thanks so much for the reply and all the information. Would you do me a favor and have a look at the esp-HomeKit-device project and let me know if what I want is even at all possible? And perhaps point me in the right direction? Thanks again!

https://github.com/RavenSystem/esp-homekit-devices

kueblc commented 5 years ago

If you can link me to a binary, I can test it on a dev board

tonysprenk commented 5 years ago

That would be great! There are 3 binaries:

https://github.com/HomeACcessoryKid/life-cycle-manager/releases/download/1.0.0/otaboot.bin

https://github.com/SuperHouse/esp-open-rtos/raw/master/bootloader/firmware_prebuilt/rboot.bin

https://github.com/SuperHouse/esp-open-rtos/raw/master/bootloader/firmware_prebuilt/blank_config.bin

As I understand it it OTAboot.bin allows the device to come online with an SSD so it can be connected to your network, I’m guessing rboot.bin is the bootloader and blank_config.bin is there for the “template” to map the buttons and relays etc.

Thank you so much for looking into this!

kueblc commented 5 years ago

Worked fine for me. You'll need to bundle these into one binary to upload in one shot.

# first ensure the bootloader is exactly one sector (4kb)
truncate -s 4k rboot.bin
# same for config
truncate -s 4k blank_config.bin
# bundle it up
cat rboot.bin blank_config.bin otaboot.bin > thirdparty.bin

Then you can replace the thirdparty.bin in your tuya-convert/files and proceed as normal.

kueblc commented 5 years ago

I noticed on the wiki page you can install main.bin instead of otaboot.bin, so you could skip a step by installing it directly, just replace otaboot with main in the steps above. I didn't read into this too much, maybe there's another reason you might want otaboot.bin.

tonysprenk commented 5 years ago

Thanks so much for looking into this and pointing me in the right direction.

So just tried this but the LCM-XXXXX network never came online, I think the plug is bricked now. Would you mind sharing your bundled binary? Maybe I did something wrong there.

Logs:

smarthack-wifi.log smarthack-web.log smarthack-psk.log smarthack-mqtt.log

kueblc commented 5 years ago

Hmm, sorry to hear that. Here's what I flashed: thirdparty.zip. You can diff this one from yours to see if there's any difference.

One thing I did notice is that the AP this firmware creates is pretty unstable, whether flashed OTA or by wire. Even with a freshly erased flash, I could not stay connected. I was able to connect long enough to see the settings page, but afterwards it did not respond to a page refresh.

Since this happens even when flashed over wire I suspect there's some issue with the current build, or perhaps some I'm missing something, init data etc.

tonysprenk commented 5 years ago

There is no difference. I just remembered that while flashing I didn't check if it was in user2 and after I flashed it it said it flashed to userspace1, could that be the problem? All my other plugs were already in user2 when I did check.

kueblc commented 5 years ago

No, it should be flashed to userspace1. The intermediate firmware won't let you flash the thirdparty.bin unless you are already running in userspace2 and will let you know otherwise.

kueblc commented 5 years ago

I wonder if maybe the flash mode/size/speed settings did not match your device. As specified in the README and during the start_flash procedure, they need to match otherwise your device may be unable to boot. To avoid issues like this in the future, this will be handled automatically in the next release (see #395).

Can you pull up your device-info.txt for this device?

tonysprenk commented 5 years ago

This wasn't an issue when converting to Tasmota or Espurna.

device-info.txt

kueblc commented 5 years ago

Indeed the flash settings do not match. Your device was set to DOUT @ 1MB while the rboot.bin is set to QIO @ 2MB. You can run DOUT on hardware that supports QIO, but not the other way around. Tasmota and Espurna on the other hand have the same settings as your device.

tonysprenk commented 5 years ago

OK well that explains a lot then. Thanks so much for checking this out. Is there anything I can do to avoid this if I want to flash another plug?

kueblc commented 5 years ago

One way, you can manually overwrite the flash settings in rboot.bin.

If you open rboot and your firmware backup in a hex editor side by side, you'll see that rboot starts with E9 02 00 30 and your firmware starts will E9 03 03 20.

The third and fourth bytes define the flash settings, so change rboot to E9 02 03 20 and save. Make sure you overwrite these bytes instead of inserting, you don't want to change anything else but those two bytes.

Alternatively, you could try #395, which as I mentioned earlier will perform this automatically. It should be merged with the next release, assuming no issues.

tonysprenk commented 5 years ago

It worked! thank you so so much! if there's anything I can do in testing or whatever just let me know!

kueblc commented 5 years ago

Awesome, so happy to hear it! Did you try #395 or do it manually?

We could always use more beta testers. Feel free to try out any of the open PRs now or in the future and leave your feedback. I really appreciate it!

You could also consider making a donation to support the project so I can continue buying equipment to test, and spread the word about tuya-convert!

seritos commented 4 years ago

Hi All

I have a Houzetek AWP07L that is really driven myself crazy. I have been trying to flash it with HAA since December. It has just become a personal challenge.

I have flashed dozens of devices and have experienced issues, but never like this one.

So the problem is that I can not make it work with HAA/Ravencore. However, it just works perfectly with tasmota. The device has the cables soldered.

So I tried:

Flash it with an slower speed: I have only eventually reached the LCM broadcast and set up the wifi and the OTA url, but when the next steps comes, the device just bricks. When the LCM doesn't came right after flashing, I can see either the led constantly blinking, or just blink once, and then turns almost off but with a very very tiny beam of light on. In both cases nothing happens and the device is also bricked...

Flash it with the file unified with the command: dd if=blank_config.bin of=newblank_config.bin bs=4k count=1 .... just the same as mentioned above but with dd instead of truncate. ... bricked result: when flashed with the command: esptool.py --port /dev/tty.usbserial-00000000 write_flash --flash_size 1MB --flash_mode dout 0x00000 xxxxx.bin

Flash it directly from tasmota update webserver: same results... nothing.

Flash with the file thirdparty.bin uploaded above: Same results.

I have not ever tried the tuya system itself, as I don't have the required HW, but I believe that with the cables should be enough...

I sometimes think that is an issue with the flashing speed, others with a memory address... I am really lost. Any clues...? It has really become a personal fight!!

Thanx!

seritos commented 4 years ago

just flashed with https://github.com/RavenSystem/esp-homekit-devices/issues/683#issuecomment-569532286. and again bricked the led flashes every second and no LCM is visible.

Now If I erase it, it tells me that the procedure has taken 0 seconds... as if memory would be empty ?¿?

Thnx

tonysprenk commented 4 years ago

try to get some logs and open a new issue on the HAA repo.

seritos commented 4 years ago

try to get some logs and open a new issue on the HAA repo.

https://github.com/RavenSystem/esp-homekit-devices/issues/778#issue-558514858

seritos commented 4 years ago

One way, you can manually overwrite the flash settings in rboot.bin.

If you open rboot and your firmware backup in a hex editor side by side, you'll see that rboot starts with E9 02 00 30 and your firmware starts will E9 03 03 20.

The third and fourth bytes define the flash settings, so change rboot to E9 02 03 20 and save. Make sure you overwrite these bytes instead of inserting, you don't want to change anything else but those two bytes.

Alternatively, you could try #395, which as I mentioned earlier will perform this automatically. It should be merged with the next release, assuming no issues.

I believe this could also be my issue. I can't test it right now, but I have the .bin files with me. I also bricked my device, although I have soldered and I am able now to unbrick it.

So the original Backup file starts in E9 03 03 20 Tasmota begins: E9 01 03 20 rboot: E9 02 00 30

I understand, that If I want to make it work, I should change the rboot to E9 03 03 to match the original FW... shouldn't I?

On the other hand, my device works great in Tasmota but not in HAA. I have eventually managed to have the LCM network on-line , but this has just happened twice in 100 tests. I have tested in this in other identical device, so I discard the option of being broken.

Thanx!!

seritos commented 4 years ago

I have tried it all. Replace just that header. Replace the hole rboot sector. Transfer it in 20 MHz, 26 Mhz... I have just no idea what else can I do. Its not even accepting the thirdparty bin from tuya and neither Espurna or any other FW.

... @kueblc Is there anything else possible to do? It only accepts the original FW and eventually the tasmota.

Thnx much