rednblkx / HomeKey-ESP32

ESP32 HomeKit Lock with support for Apple Home Key
MIT License
421 stars 18 forks source link

Can't get Wifi connected, ESP rebooting constantly #62

Closed x23piracy closed 3 months ago

x23piracy commented 3 months ago

Hi,

i compiled and flashed on a ESP32-WROOM-32, it's booting but there is constantly output flowing over the screen i cannot press W + Enter or A nor it's doesn't react to the cables shortended between GND and GPIO26, what i am doing wrong?

I can also upload pictures and or console output if i am able to capture because its constaly repeating things.

EDIT: this repeats constantly in serial console:

` I (1020) SETUP: setup > HOMEKEY ISSUERS: 0


                                                        Welcome to HomeSpan!
                                                                            Apple HomeKit for the Espressif ESP-32 WROOM and Arduino IDE
                                                                                                                                        ************************************************************

                                                                                                                                                                                                    ** Please ensure serial monitor is set to transmit <newlines>

                                                                                                                                                                                                                                                                 Message Logs:     Level 0
                                                                                                                                                                                                                                                                                          Status LED:       Pin 2  (Auto Off=15 sec)
       Device Control:   Pin 26
                               Sketch Version:   1
                                                  HomeSpan Version: 1.9.0
                                                                         Arduino-ESP Ver.: 2.0.16
                                                                                                 ESP-IDF Version:  4.4.7
                                                                                                                        ESP32 Chip:       ESP32-D0WD-V3 Rev 3 dual-core 4MB Flash
                                                                                                                                                                                 ESP32 Board:      esp32
                                                                                                                                                                                                        PWM Resources:    16 channels, 8 timers, max 20-bit duty resolution
                                                                                                                                                                                                                                                                           Sodium Version:   1.0.12-idf  Lib 9.4
                                                                                                                                                                                                                                                                                                                MbedTLS Version:  Mbed TLS 2.28.7
                    Sketch Compiled:  Jun  6 2024 22:40:50
                                                          Partition:        app0I (3088) system_api: Base MAC address is not set

I (3089) system_api: read default base MAC address from EFUSE

MAC Address: 0C:B8:15:XX:XX:XX

                               Device Name:      HK Lock

                                                        ets Jul 29 2019 12:21:46

rst:0x8 (TG1WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0030,len:5596 ho 0 tail 12 room 4 load:0x40078000,len:15860 load:0x40080400,len:3624 entry 0x4008064c W (172) boot.esp32: PRO CPU has been reset by WDT. W (172) boot.esp32: WDT reset info: PRO CPU PC=0x40082ba1 W (174) boot.esp32: WDT reset info: APP CPU PC=0x40089ff8 I (790) cpu_start: Multicore app I (790) cpu_start: Pro cpu up. I (791) cpu_start: Starting app cpu, entry point is 0x40082150 I (0) cpu_start: App cpu up. I (808) cpu_start: Pro cpu start user code I (809) cpu_start: cpu freq: 240000000 I (809) cpu_start: Application information: I (813) cpu_start: Project name: HomeKey-ESP32 I (819) cpu_start: App version: 1 I (823) cpu_start: Compile time: Jun 6 2024 22:37:34 I (829) cpu_start: ELF file SHA256: c0fac961be02dcb6... I (835) cpu_start: ESP-IDF: 4.4.7 I (840) cpu_start: Min chip rev: v0.0 I (844) cpu_start: Max chip rev: v3.99 I (849) cpu_start: Chip rev: v3.0 I (854) heap_init: Initializing. RAM available for dynamic allocation: I (861) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM I (867) heap_init: At 3FFBB650 len 000249B0 (146 KiB): DRAM I (874) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM I (880) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM I (886) heap_init: At 40098CA8 len 00007358 (28 KiB): IRAM I (894) spi_flash: detected chip: gd I (897) spi_flash: flash io: dio I (906) cpu_start: Starting scheduler on PRO CPU. I (0) cpu_start: Starting scheduler on APP CPU. II (929) gpio: GPIO[26]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0 I (930) gpio: GPIO[2]| InputEn: 1| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0

Generating SRP verification data for new Setup Code: 466-37-726 ... New Code Saved! Setup Payload for Optional QR Code: X-HM://00057L40U

`

It looks like its rebooting all the time.

W (172) boot.esp32: PRO CPU has been reset by WDT. W (172) boot.esp32: WDT reset info: PRO CPU PC=0x40082ba1 W (174) boot.esp32: WDT reset info: APP CPU PC=0x40089ff8

Best Regards X23

011V32 commented 3 months ago

You need to configure your serial terminal to the right line ending. (I think it is /n only, but look in the documentation of Homespan.)

x23piracy commented 3 months ago

Hi,

yes you saw it used 115000 instead of 115200 anway this changes nothing the output is still scrolling and i looks like its resetting all the time, it doesnt react to my input.

W (172) boot.esp32: PRO CPU has been reset by WDT. W (172) boot.esp32: WDT reset info: PRO CPU PC=0x40082ba1 W (174) boot.esp32: WDT reset info: APP CPU PC=0x40089ff8

https://github.com/orgs/micropython/discussions/10377 https://github.com/orgs/micropython/discussions/10377#discussioncomment-6439625

Best Regards X23

rednblkx commented 3 months ago

Yes, the line ending has nothing to do with the reboot, the ESP is being resetted by the Watchdog Timer for some reason.

The thread you linked is related to a wifi issue (possibly specific only to micropython), however wifi is handled entirely by HomeSpan which i haven't yet seen to cause that issue.

I see you have an ESP32-D0WD-V3 which should work without issues as it's the same version i've used. Could you share what board are you using? If it's an MH-ET D1 Mini that should also not have issue in which case and if erasing the flash and doing a clean build does not help then you might have a bad board.

There are some changes in the dev branch that might help, please try it and see if it still behaves the same, make sure to erase the flash before flashing. If it's still crashing then i can provide you with a known working firmware file just to see if something strange is going on on your end.

x23piracy commented 3 months ago

Sure here is a picture, got 3 and i already used one of them for ahoy dtu.

image

Well even with the massive (lol) cap it's rebooting, the cap is btw. to fat had no other one around :D I have currently a RC522 in use and i was searching for home key with wemos d1 but it seems that i at least have to use an esp32, i also found a project with a rpi but this to much for that little thing and i have not enough power at the frontdoor for driving a pi.

@rednblkx i'll try the dev branch and report.

011V32 commented 3 months ago

The reboot not, but that it didn't react to the Input could be the line ending. I had the same problem... The Terminal on windows needs to have LF also for transmitting (Putty only over Ctrl+J instead Enter, so I choosed Tera Term instead)

rednblkx commented 3 months ago

@x23piracy the RC522 is not supported, you need a PN532 for it to work. Also note that even though esp32 is low power compared to rpi it will still use some amount of power that it won't be suitable to be powered by a battery long-term if you were planning on doing that as it doesn't sleep due to HomeKit needing a persistent connection.

rednblkx commented 3 months ago

The reboot not, but that it didn't react to the Input could be the line ending. I had the same problem... The Terminal on windows needs to have LF also for transmitting (Putty only over Ctrl+J instead Enter, so I choosed Tera Term instead)

That yes though what you are referring is what windows uses as a standard which is actually CRLF not LF, the LF ending is used by Unix systems, anyhow does not matter for Serial as that's set by the device not on host which in this case is CRLF for receive and LF for transmit which are esp-idf defaults.

x23piracy commented 3 months ago

Hi,

i just told ya i have currently a RC522 in place with a wemos, the ESP32 is complete other thing i have everything here so all cool.

IMG_1483

I tried the dev branch:

grafik

x23piracy commented 3 months ago

Funny i flashed the non branch again and now it works :D well the HK Lock doesn't react yet even when the key is detected.

rednblkx commented 3 months ago

Glad it works now, though very strange.

Note that the key and the lock are not coupled together, they are two separate things and by default an authentication event does not lock or unlock the "HK Lock" directly with only some cases where it does as detailed in the wiki.

This has slightly changed in the dev branch along implementing support for gpio status and action pins, more will be detailed in the wiki once released.

for the error above with the dev branch, it's because i put / in the spiffs image location as i use Linux and Windows uses \ as a path separator, will be fixed once merged into main. The dev branch now implements a webui which has the html files stored in the spiffs partition and it is flashed separately hence the spiffs image.

In the meantime if you still want to try it, you can open the fs.py and just replace the / in $BUILD_DIR/$ESP32_FS_IMAGE_NAME or delete the following line from platformio.ini to stop the script from running since we are only testing to see if it "boots" extra_scripts = pre:extra_script.py, pre:fs.py

x23piracy commented 3 months ago

i just played around with what i have now at you need to really come close to the 532, the card and tag which have been shipped with the 532 have a much better distance, detected around 3-4cm away from the board, i think this is by design (apple)?

x23piracy commented 3 months ago

It would be really cool if this in the end will have a webif where we can add tags that are allowed, i did this with tasmota and the RC522 combined with rules, i wrote a salt into block01 to not just have the uid which maybe can be easily bruteforced (4 times a second) therefore i created rules within tasmota which check for uid and my salt if both match i open the lock which is currently made with homebridge and http lock plugin :) a lot of combining this and that :D

B2980D70-2988-4AEB-875A-E8901EF11550 IMG_1389

IMG_1485

all 3d printed and painted on my own.

rednblkx commented 3 months ago

i just played around with what i have now at you need to really come close to the 532, the card and tag which have been shipped with the 532 have a much better distance, detected around 3-4cm away from the board, i think this is by design (apple)?

yes and no, you can actually get 5-6cm if you have a reader with a proper antenna, that's why it's best to get the original module from Elechouse if possible, though, i was looking at ways to shield from EMI and place the reader on metal surface and discovered you can place piece of ferrite sheet and aluminium foil on the back of the reader and it focuses the field better and as a result increases the range a bit to like 5-6 cm or at least it worked that way with one of the modules i bought.

Also if you have an apple watch, you might notice that it has a better reception and that's because compared to an iphone which has a small antenna in the left corner, the watch has a bigger antenna the size of screen

It would be really cool if this in the end will have a webif where we can add tags that are allowed, i did this with tasmota and the RC522 combined with rules, i wrote a salt into block01 to not just have the uid which maybe can be easily bruteforced (4 times a second) therefore i created rules within tasmota which check for uid and my salt if both match i open the lock which is currently made with homebridge and http lock plugin :) a lot of combining this and that :D

that's really cool, i do plan to add whitelisting tags and possibly some other stuff but i also want to make it as stable as possible and also more secure.

rednblkx commented 3 months ago

i just recently implemented a basic webui that's still in the dev branch so there is a long way

x23piracy commented 3 months ago

Cool, i stay tuned and will look for this project :) no hurry i have a solution in place but like the idea to use home key, so what i found is that you need to authenticate against the tag depending on the technology (mifare etc.) then you are able to write data to block 01, is the PN532 able to write or can it just read?

i have a apple watch too, a gen 0 one :D the first one which lost support weeks ago, no more spare part :D it currently loading will test if the gen 0 will work with it.

x23piracy commented 3 months ago

hmm the home key doenst seem to appear on the old apple watch, meh series 4 and up. I get me a new one :D

rednblkx commented 3 months ago

is the PN532 able to write or can it just read?

the module itself can of course do both, however, this project currently only reads the uid the tag and publishes it via MQTT.

what i found is that you need to authenticate against the tag depending on the technology (mifare etc.) then you are able to write data to block 01

yes, cards like mifare classic for example, have a series of keys for each their sectors(which contain the blocks) and without keys you can't read the sectors. At some point i am thinking of adding stuff like that so you can have options but for the moment trying to figure out the structure of the whole thing.

hmm the home key doenst seem to appear on the old apple watch, meh series 4 and up. I get me a new one :D

well yeah, homekey is relatively new, ios 15

x23piracy commented 3 months ago

can also publish data thats what i wrote to block 01, if i hold it to the RC522 it output uid and data

if have the following hw too:

243EB4A2-16FF-413E-9858-9931B0AC0B68 BDA8F61F-8B0D-4EBC-B830-797A9D84B40E

this happens when key is not enough, rfid, nfc, fingerprint, siri, face recog. In the end using the foot is also an option πŸ˜‚

x23piracy commented 3 months ago

what is the real front of the eletech module? the site with the led where the antenna is printed on the pcb? if so it maybe makes sense to solder the cable directly from the backside, maybe the connector interference the communication, i recieve really often an error try again.

i have a second module here i can try with. How exactly place a shield of metal? do you have a picture?

How can i make the HK Lock open/close when i test the home key functionality? Do i understand correctly, i have to enable:

MQTT_CUSTOM_STATE_ENABLED to 1 MQTT_HOMEKEY_ALWAYS_UNLOCK to 1

(i want that the nuki unlocks if locked and open the trap and if already unlocked just pull the trap.)

rednblkx commented 3 months ago

what is the real front of the eletech module?

if by "real front" you mean which way is the antenna facing then none as that's not how the RF field works, should work the same both ways as long as there is no interference

How exactly place a shield of metal? do you have a picture?

try something like this

IMG_3427 IMG_3428

How can i make the HK Lock open/close when i test the home key functionality?

You publish the state that you want to the MQTT topic MQTT_SET_STATE_TOPIC

Do i understand correctly, i have to enable:

MQTT_CUSTOM_STATE_ENABLED to 1 MQTT_HOMEKEY_ALWAYS_UNLOCK to 1

(i want that the nuki unlocks if locked and open the trap and if already unlocked just pull the trap.)

Someone actually opened an issue a while back with the idea of controlling directly a Nuki lock via its MQTT Api which led to the implementation of mqtt custom states.

So ....yeah, if you want to talk "directly" to the lock, set MQTT_CUSTOM_STATE_ENABLED to 1 or true then set MQTT_CUSTOM_STATE_TOPIC to Nuki's control topic and MQTT_CUSTOM_STATE_CTRL_TOPIC to Nuki's state topic so you can keep "HK Lock" in sync. Additionally take a look at customLockStates, there you can assign the state numbers from Nuki's documentation to what you want e.g nuki latched state linked to homekit locked

x23piracy commented 3 months ago

just tried apple watch se from my wife, not successful just states try again. I had 4 attempts, now shes gone with her watch :D as i can see you had the same idea solder from behind. i'll try with my second one :) i will create a 3d printed case for both the esp32 combined with the 532 and a slot for a metal plate, maybe i try aluminum?

how did you isolate against shorts? looks like double sided foam tape, isn't it? am i able to change the gpios? so i can maybe solder a solid 4 pin slot unter the 532 so i can connect everything expect vcc and ground by a stick connection. what a pitty no way to assemble both without the use of cable.

Maybe its better to keep the esp32 away from the reader, maybe interferences or so?

https://www.printables.com/de/model/612413-esp32-case/files#preview this seems to be a good thing to start with.

uhhh better: https://www.printables.com/de/model/531938-esp32-pn532 not sure this fits in the room i have under the doorbell, need to measure, maybe i can reduce it

rednblkx commented 3 months ago

just tried apple watch se from my wife, not successful just states try again. I had 4 attempts, now shes gone with her watch :D

trial over, now buy your own :))) joke aside, i think it might have failed provisioning so it's missing ids and keys. Try the following:

how did you isolate against shorts? looks like double sided foam tape, isn't it?

It's not, i told you, it's ferrite sheet(used magnetic sheet for photos), stacked on top of each other, the ferrite acts as an electromagnetic barrier to shield the antenna from interference and combined with the aluminium foil blocks the rf from the block and just bounces back i guess :shrug: , don't know exactly the whole process for the aluminium foil.

am i able to change the gpios? so i can maybe solder a solid 4 pin slot unter the 532 so i can connect everything expect vcc and ground by a stick connection. what a pitty no way to assemble both without the use of cable.

power doesn't flow through the spi pins so you have to power the nfc module through its vcc and gnd pins

Maybe its better to keep the esp32 away from the reader, maybe interferences or so?

i think generally it's better to keep them away from each other, however, i still just stacked mine πŸ˜… as i wanted to print a small square case.

x23piracy commented 3 months ago

I tried aluminum/thin double sided tape/paper, decreases distance heavy.

image

lol while i took that foto the phone was at least 12cm away from the module the key took place πŸ˜‚

tried again, wow its more then 12cm

rednblkx commented 3 months ago

not sure how you did that but i'm glad it worked out for you, that's hilarious πŸ˜‚

x23piracy commented 3 months ago

It seems to be the position or earth radiation idk :D i try to make a video of it, phone is an iphone 13

rednblkx commented 3 months ago

the issue is that per your description, you don't have a ferrite layer between the module and the aluminium foil. If you only have the aluminum foil, it absorbs the magnetic field, but idk maybe there is something in your double sided tape πŸ˜‚

rednblkx commented 3 months ago

not an expert however, just stuff i've found during my research and what i've observed during my tinkering

x23piracy commented 3 months ago

is it true that homespan can be used for later ota updates? i like to change the RC522 against the PN532 but its no fun to access the usb port :D

x23piracy commented 3 months ago

what do you mean with control and state topic, state is a match but control?

grafik

i can see without enabling anything it's quiet silent here:

grafik

rednblkx commented 3 months ago

what do you mean with control and state topic, state is a match but control?

the control topic is where you send your command to the lock and the state topic is where you receive the current state of the lock

based on Nuki's MQTT API documentation, the state topic should be nuki/hex_id/state and the control topic should be nuki/hex_id/lockAction

So then the configuration would be like this:

i can see without enabling anything it's quiet silent here:

homekey_mqtt is an "internal topic" and only contains status as it only serves as LWT and availability topic for HASS as it requires that.

You need to look into the topics you have defined in the "MQTT Topics" section

x23piracy commented 3 months ago

Hi thanks for helping here out, that works well i also enabled MQTT_HOMEKEY_ALWAYS_UNLOCK 1 so it will just always open the lock but doesnt pull the trap how can i archive another lock state when unlocking or in the event it is already open and need to open the trap.

its lockAction 2 and 3

it think i have to adjust UNLOCK to 3.

{ UNLOCK = 3, LOCK = 2 };

grafik

what about this simple lockactions, i can't see a topic like that in mqtt explorer for the nuki, i just ask because there is also a unlatch (thats what i was calling pulling the trap).

German front doors :)

It works now 😁 I ended up like this:

image

image

Btw. thanks for the project it really works nice as i understood how to use it, i will now change my RC522 against the PN532 and will also automate the alarm against the lock state if all 4 iphones left the building.

Is there a way to disable the led on the PN532 board and maybe use one of the gpio pins on the esp32 for an alternative led? is it maybe also possible to connect a beeper for an acustic feedback?

Wishlist:

Lol so much wishes, whats your paypal :D My issue is solved and can be marked as those, is there another way to communicate with you?

Btw. one of my 3 esp32 seem to be broken in some way, at the beginning i seem have to swapped the broken against one of the other 3 this was the reason it started working, maybe i was to stoned to realize i changed the esp32 lol :)

rednblkx commented 3 months ago

Is there a way to disable the led on the PN532 board and maybe use one of the gpio pins on the esp32 for an alternative led? is it maybe also possible to connect a beeper for an acustic feedback?

The LED on the PN532 board is hardwired so you'll have to desolder it. Regarding assigning a gpio pin on the esp32 for an led or a buzzer, that is one of the things that is already implemented in the dev branch

Wishlist:

* OTA Update

* kept settings

* webif with all that incredible options to change :P

* RFID/NFC dongle support (Whitelist), maybe add Security (Block 01)
  1. OTA updates already present as part of HomeSpan, keep forgetting to add entry about it on the wiki, i only did a short paragraph on the README
  2. Not sure i understand, could please give me more details on this?
  3. A basic web interface has already been implemented in the dev branch with all the current options from config.h and a little more
  4. Tag whitelisting is on the map but i don't currently plan on implementing this very soon

Once the dev branch is ready, i'll have updated the wiki pages and be merged into main with an entry in the releases page detailing all that has chanegd.

Lol so much wishes, whats your paypal :D My issue is solved and can be marked as those, is there another way to communicate with you?

I don't use paypal but you can choose to sponsor me on github if you wish to do so and can contact me by email on rednblkx@protonmail.com or on Discord @rednblkx

Since the original problem has been fixed(or rather fixed itself, which is always nice 🫠), i'll be closing this issue now, if you have a subject to talk about, a question or a suggestion, you can also create a new discussion in the discussions section.

x23piracy commented 3 months ago

For 2. i ment making the options changeable via webif persistently. I'll try dev again with the backup devices i have, i can then also test ota with it before i always need to disassemble the front door setting.

It seems that i have a little WLAN issue with the esp32 at the front door with a repeater right behind the wall 2m airline, sometimes there is a delay when opening the door, even not so much like with the fingerprint reader but there is one from time to time.

x23piracy commented 3 months ago

i tried current dev but after flashing i get:

[22:11:56]rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) [22:11:56]configsip: 0, SPIWP:0xee [22:11:56]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 [22:11:56]mode:DIO, clock div:2 [22:11:56]load:0x3fff0018,len:4 [22:11:56]load:0x3fff001c,len:1044 [22:11:56]load:0x40078000,len:8896 [22:11:56]load:0x40080400,len:5828 [22:11:56]entry 0x400806ac [22:11:57]ets Jul 29 2019 12:21:46

just this in a loop

rednblkx commented 3 months ago

Looks like it didn’t flash right and it’s not able to boot the firmware so it just keeps resetting.

I know you’ve had an error previously with the spiffs, how did you handle it?

x23piracy commented 3 months ago

I used ESPHome Flasher for the bin, i should try with vcode but i still get an error, i replaced / against \

grafik

grafik

\littlefs\.bin' that part is confusing, there is no such directory and it should be firmware.bin right?

rednblkx commented 3 months ago

well there's the issue, you're missing partitions, the bootloader, partitions header and ota_data

The thing with the ESP32 toolchain is that once the code is compiled, you get 4 .bin files(or more depending on partitions configuration), bootloader.bin, firmware.bin where the actual code lives, partitions.bin which defines the structure and an ota_data_initial.bin which stores data for the current running partition.

The tool that builts littlefs.bin might be missing on windows or something, this only stores the html files for the webui, i'll take a look when i can to see what's going on with Windows, in the meantime if you want you can flash the image i've attached below, i've merged the images into one, just use esptool.py like so esptool.py write_flash 0x0 hk_dev.img and that should get you up and running.

hk_dev.bin.zip

x23piracy commented 3 months ago

Hi, thanks so far but it get:

Unexpected error: The firmware binary is invalid (magic byte=FF, should be E9)

used esptool:

load:0x40078000,len:14672 ho 0 tail 12 room 4 load:0x40080400,len:3460 entry 0x40080638 E (773) esp_image: Image length 1967072 doesn't fit in partition length 1966080 E (773) boot: OTA app partition slot 0 is not bootable E (776) esp_image: image at 0x200000 has invalid magic byte (nothing flashed here?) E (784) boot: OTA app partition slot 1 is not bootable E (790) boot: No bootable app partitions in the partition table ets Jul 29 2019 12:21:46

rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) configsip: 0, SPIWP:0xee clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00 mode:DIO, clock div:2 load:0x3fff0030,len:5428 load:0x40078000,len:14672 ho 0 tail 12 room 4 load:0x40080400,len:3460