search

esphome / firmware

Holds firmware configuration files for projects that the ESPHome team provides.
https://esphome.io/projects
Apache License 2.0
141 stars 109 forks source link

M5 Stack Atom Echo VA firmware does not install locally since a79c9fa - image too big. #227

Open HarvsG opened 2 weeks ago

HarvsG commented 2 weeks ago

Environment: ESPHome 2024.6.6 on HA OS. Raspberry Pi 4 - I can also re-create the issue in docker 2024.6 on windows. M5 Stack Atom Echo.

Related: https://github.com/esphome/issues/issues/5714#issuecomment-2211542063 Another user with the same issue: https://community.home-assistant.io/t/m5-stack-atom-echo-installation/746641/2

I can install the VA config with Ready-made projects however when I try and adopt the device locally, or even manually add the device and install the VA config

substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom
packages:
  m5stack.atom-echo-voice-assistant: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: true
  friendly_name: ${friendly_name}
api:
  encryption:
    key: someAPIkey=
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

I get the following on loop.

INFO Starting log output from /dev/ttyUSB1 with baud rate 115200
[13:02:07]8069c
[13:02:07]I (29) boot: ESP-IDF 4.4.7 2nd stage bootloader
[13:02:07]I (30) boot: compile time 12:57:16
[13:02:07]I (30) boot: Multicore bootloader
[13:02:07]I (33) boot: chip revision: v1.1
[13:02:07]I (36) boot.esp32: SPI Speed      : 40MHz
[13:02:07]I (41) boot.esp32: SPI Mode       : DIO
[13:02:07]I (46) boot.esp32: SPI Flash Size : 4MB
[13:02:07]I (50) boot: Enabling RNG early entropy source...
[13:02:07]I (56) boot: Partition Table:
[13:02:07]I (59) boot: ## Label            Usage          Type ST Offset   Length
[13:02:07]I (66) boot:  0 otadata          OTA data         01 00 00009000 00002000
[13:02:07]I (74) boot:  1 phy_init         RF data          01 01 0000b000 00001000
[13:02:07]I (81) boot:  2 app0             OTA app          00 10 00010000 001c0000
[13:02:07]I (89) boot:  3 app1             OTA app          00 11 001d0000 001c0000
[13:02:07]I (96) boot:  4 nvs              WiFi data        01 02 00390000 0006d000
[13:02:07]I (104) boot: End of partition table
[13:02:07]I (108) boot: No factory image, trying OTA 0
[13:02:07]I (113) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=65984h (416132) map
[13:02:07]I (272) esp_image: segment 1: paddr=000759ac vaddr=3ffbdb60 size=051bch ( 20924) load
[13:02:07]I (281) esp_image: segment 2: paddr=0007ab70 vaddr=40080000 size=054a8h ( 21672) load
[13:02:07]I (290) esp_image: segment 3: paddr=00080020 vaddr=400d0020 size=1369a0h (1272224) map
[13:02:07]I (750) esp_image: segment 4: paddr=001b69c8 vaddr=400854a8 size=19adch (105180) load
[13:02:07]E (794) esp_image: Image length 1836240 doesn't fit in partition length 1835008
[13:02:07]E (794) boot: OTA app partition slot 0 is not bootable
[13:02:07]E (796) esp_image: image at 0x1d0000 has invalid magic byte (nothing flashed here?)
[13:02:07]E (805) boot: OTA app partition slot 1 is not bootable
[13:02:07]E (810) boot: No bootable app partitions in the partition table
[13:02:07]ets Jun  8 2016 00:22:57
[13:02:07]
[13:02:07]rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[13:02:07]configsip: 188777542, SPIWP:0xee
[13:02:07]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[13:02:07]mode:DIO, clock div:2
[13:02:07]load:0x3fff0030,len:6652
[13:02:07]ho 0 tail 12 room 4
[13:02:07]load:0x40078000,len:15052
[13:02:07]load:0x40080400,len:3836
[13:02:07]entry 0x4008069c
[13:02:07]I (57) boot: ESP-IDF 4.4.7 2nd stage bootloader
[13:02:07]I (57) boot: compile time 12:57:16
[13:02:07]I (58) boot: Multicore bootloader
[13:02:07]I (61) boot: chip revision: v1.1
[13:02:07]I (65) boot.esp32: SPI Speed      : 40MHz
[13:02:07]I (70) boot.esp32: SPI Mode       : DIO
[13:02:07]I (74) boot.esp32: SPI Flash Size : 4MB
[13:02:07]I (79) boot: Enabling RNG early entropy source...
[13:02:07]I (84) boot: Partition Table:
[13:02:07]I (88) boot: ## Label            Usage          Type ST Offset   Length
[13:02:07]I (95) boot:  0 otadata          OTA data         01 00 00009000 00002000
[13:02:07]I (103) boot:  1 phy_init         RF data          01 01 0000b000 00001000
[13:02:07]I (110) boot:  2 app0             OTA app          00 10 00010000 001c0000
[13:02:07]I (118) boot:  3 app1             OTA app          00 11 001d0000 001c0000
[13:02:07]I (125) boot:  4 nvs              WiFi data        01 02 00390000 0006d000
[13:02:07]I (133) boot: End of partition table
[13:02:07]I (137) boot: No factory image, trying OTA 0
[13:02:07]I (142) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=65984h (416132) map
[13:02:08]I (301) esp_image: segment 1: paddr=000759ac vaddr=3ffbdb60 size=051bch ( 20924) load
[13:02:08]I (310) esp_image: segment 2: paddr=0007ab70 vaddr=40080000 size=054a8h ( 21672) load
[13:02:08]I (319) esp_image: segment 3: paddr=00080020 vaddr=400d0020 size=1369a0h (1272224) map
[13:02:08]I (779) esp_image: segment 4: paddr=001b69c8 vaddr=400854a8 size=19adch (105180) load
[13:02:08]E (823) esp_image: Image length 1836240 doesn't fit in partition length 1835008
[13:02:08]E (823) boot: OTA app partition slot 0 is not bootable
[13:02:08]E (825) esp_image: image at 0x1d0000 has invalid magic byte (nothing flashed here?)
[13:02:08]E (834) boot: OTA app partition slot 1 is not bootable
[13:02:08]E (839) boot: No bootable app partitions in the partition table
[13:02:08]ets Jun  8 2016 00:22:57
[13:02:08]
[13:02:08]rst:0x3 (SW_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[13:02:08]configsip: 188777542, SPIWP:0xee
[13:02:08]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[13:02:08]mode:DIO, clock div:2
[13:02:08]load:0x3fff0030,len:6652
[13:02:08]ho 0 tail 12 room 4
[13:02:08]load:0x40078000,len:15052
[13:02:08]load:0x40080400,len:3836
[13:02:08]entry 0x4008069c

logs_m5-atom-echo_run (1).txt

I can install other configs e.g a plain ESP Home web, and the user above has had success with another VA config, which is what makes me think it is an issue with this particular firmware.

lmaso90 commented 2 weeks ago

I also have the same issue

HarvsG commented 2 weeks ago

I think the issue might be https://github.com/esphome/firmware/commit/a79c9fa96fe0fdd24c17e7571de3ef37755d54a1 / #223 because rolling back worked github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@cd57ca6f951d44cc5bf61de124a15b349ef1f9a4

substitutions:
  name: m5stack-atom-echo-b836b0
  friendly_name: M5Stack Atom Echo b836b0
packages:
  m5stack.atom-echo-voice-assistant: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@cd57ca6f951d44cc5bf61de124a15b349ef1f9a4
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: +myAPIKEY=

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
TxHarleyDude commented 2 weeks ago

I followed the suggestion from HarvsG and it got me back working as well.

My Setup: Home Assistant: 2024.7.1 ESPHome: 2024.6.6

My issues started when ESPHome upgraded to 2024.6.6 Same error as above.

lmaso90 commented 2 weeks ago

The suggestion from HarvsG also worked for me

molusk commented 2 weeks ago

Thx for the workaround @HarvsG 👍

SDPismo commented 2 weeks ago

Rollback worked for me. Thx! Wasted a lot of time working on this before giving up and rolling back :/ Looking forward to seeing what happened and a solution other than the workaround.

HAOS on Raspberry Pi 5 8GB Ram Core version 2024.7.2 Supervisor 2024.6.2 OS 6.4 ESPHome 2024.6.6

HarvsG commented 2 weeks ago

Tagging @jesserockz as he made the commit and it seems this error is very reproduceable. Apologies for the @.

HarvsG commented 2 weeks ago

So I have done a bit of work to narrow down the issue and this also works as a workaround. (Edit: some are reporting limited success) Edit: clean your build files before trying this workaround

substitutions:
  name: m5stack-atom-echo-b836b0
  friendly_name: M5Stack Atom Echo b836b0
packages:
  m5stack.atom-echo-voice-assistant: github://esphome/firmware/voice-assistant/m5stack-atom-echo.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false
  friendly_name: ${friendly_name}
api:
  encryption:
    key: +yourAPIKEY=

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

# HarvsG's customisations 2024-07-10, will likely break UI updating in HomeAssistant and 
# can be deleted once https://github.com/esphome/firmware/issues/227 is fixed
update:
  - platform: http_request
    id: !remove update_http_request
almoney commented 2 weeks ago

Also how do we do real regression testing on these releases. I am going to skip all updates until I need something or a security fix is needed based on the last 2 updates...

Kaldek commented 1 week ago

Also how do we do real regression testing on these releases. I am going to skip all updates until I need something or a security fix is needed based on the last 2 updates...

Amen to that. Burned a week on this darned issue, right when I was doing some heavy lifting on voice assistants around the house.

Details of my environment:

Resolution of the above issue required that the YAML for the M5 Atom was changed for the ota component.

Previous OTA YAML that worked:

ota:
  platform: esphome
  password: !secret ota_password

Updated YAML requirement:

ota:
  - platform: esphome
    password: !secret ota_password

Right after this issue was resolved, the problem with being unable to OTA the M5 Atom started. Could be coincidental, but this is what I experienced.

Multiple attempts to flash the device via USB were made as well, but I kept getting boot loops. See attached logs. logs.txt

HarvsG commented 1 week ago

Also how do we do real regression testing on these releases. I am going to skip all updates until I need something or a security fix is needed based on the last 2 updates...

Amen to that. Burned a week on this darned issue, right when I was doing some heavy lifting on voice assistants around the house.

I have no idea what testing is done. But to be fair to the team the fact that the ready-made-project install works suggests that this error may be environment specific. E.g some on discord are not having the same issue. Edit: although he was able to later re-create the issue

It would be helpful if everyone who has experienced this issue comments (or ideally edits their comment above) to explain their environment and ESPhome version.

HarvsG commented 1 week ago

~Ok, I might have another work-around. So simply pasting the yaml from https://github.com/esphome/firmware/blob/main/voice-assistant/m5stack-atom-echo.yaml or more specifically https://raw.githubusercontent.com/esphome/firmware/main/voice-assistant/m5stack-atom-echo.yaml seems to work. This is weird because it should be exactly the same as using the default package import method. The only thing I can think of is that the package import results in an extra module being installed which tips the image into being too big.~

This only worked until I added the WiFi and API encryption.

jolbi commented 1 week ago

I am experiencing the same problem as HarvsG. I managed to flash it via esphome cli on Debian trough USB port. But after compilation is succesfull I am getting this error no matter the config I use:

ERROR Running command failed: Unable to verify flash chip connection (No serial data received.).
ERROR Please try running esptool.py --before default_reset --after hard_reset --baud 460800 --port /dev/ttyUSB0 --chip esp32 write_flash -z --flash_size detect 0x10000 /home/timg/esp_compiles/atom_echo_mostlycris/.esphome/build/m5stack-atom-echo/.pioenvs/m5stack-atom-echo/firmware.bin 0x1000 /home/timg/esp_compiles/atom_echo_mostlycris/.esphome/build/m5stack-atom-echo/.pioenvs/m5stack-atom-echo/bootloader.bin 0x8000 /home/timg/esp_compiles/atom_echo_mostlycris/.esphome/build/m5stack-atom-echo/.pioenvs/m5stack-atom-echo/partitions.bin 0xe000 /home/timg/.platformio/packages/framework-arduinoespressif32/tools/partitions/boot_app0.bin locally.
INFO Upload with baud rate 460800 failed. Trying again with baud rate 115200.

With baud rate 115200 upload is successful via USB, but: My Atom Echo is no longer recognised on USB port on a Windows PC, so I cannot/don't know how to flash it via ready-made projects. The device is also not listed in the esphome dashboard on HA anymore. But it is available as a device in HA and it is working properly.

Any ideas what could I do to bring it to initial state? Should I try to erase flash via esptool.py?

freeDom- commented 1 week ago

I am experiencing the same issue.

Details of my environment:

Rolling back the mentioned commit worked for me as workaround.

Christoph-Wagner commented 1 week ago

Same issue

Notably, id: !remove update_http_request did not work for me (error about update_http_request missing or something), and I had to instead used the rollbacked commit, which worked fine.

orywig commented 5 days ago

I am seeing this same error after I try to update my atom from ESPHome add-on to HA. Flashing the device from the ESPHome ready-made projects page works, but once I adopt it again and try to install the config from ESPHome add-on, I get the message that the OTA partition is not big enough for the package and that I need to flash it over USB instead of wirelessly. I get the same error denoted in the original post above in the logs.

image

I was able to update to the last version of 2024.6 with no issue.

CoolSaet commented 4 days ago

Same issue

  • HA OS in a Proxmox VM, all current versions.

    • 2 GB RAM, 2 cores of J4125

  • ESPHome version 2024.6.6

  • Originally installed via web, later adopted to ESPHome, which is when the error occurred.

Notably, id: !remove update_http_request did not work for me (error about update_http_request missing or something), and I had to instead used the rollbacked commit, which worked fine.

I had the same error about update_http_request missing. Cleaning the build files resolved that.

EnumC commented 4 days ago

Anyone know why NVS has size of 0006d000 (436kb)? If we reduce the partition by 4kb and increase each OTA app partition by 2kb, that would give plenty of space for the app to load.

EnumC commented 4 days ago

Actually it looks like there's unutilized space between end of phy-init (0xc000) to start of ota-0 (0x10000). That seems to be an extra 16kb we can use for our app.

mithomas commented 2 days ago

@Christoph-Wagner: I had the same issue with version 2024.7.0; with the following I could get past the dependency issue:

ota:
  - platform: http_request
    id: !remove ota_http_request

update:
  - platform: http_request
    id: !remove update_http_request

Image is still too large for installing, though.

SDPismo commented 2 days ago

Does anyone know if any developer is looking into this issue yet? Seems like it's wide spread and still occurring. As mentioned in another thread, I didn't see anything that appeared to address this in the latest esphome release (2024.7.1).

HarvsG commented 2 days ago

Does anyone know if any developer is looking into this issue yet? Seems like it's wide spread and still occurring. As mentioned in another thread, I didn't see anything that appeared to address this in the latest esphome release (2024.7.1).

No, it doesn't look as if it has got any traction

In the meantime, I've opened a PR #242 that supports optional on-device wake words, you can use this config to install it. Note, don't 'adopt' the device as that will overwrite my version until #242 is merged

substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom Echo
  micro_wake_word_model: okay_nabu # hey_jarvis and hey_mycroft are also supported
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://HarvsG/firmware/wake-word-voice-assistant/m5stack-atom-echo.yaml@patch-3
esphome:
  name: ${name}
  name_add_mac_suffix: True
  friendly_name: ${friendly_name}
api:
  encryption:
    key: SOMEAPIKEY
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Enjoy!

KEV-github commented 2 days ago

Does anyone know if any developer is looking into this issue yet? Seems like it's wide spread and still occurring. As mentioned in another thread, I didn't see anything that appeared to address this in the latest esphome release (2024.7.1).

No, it doesn't look as if it has got any traction

In the meantime, I've opened a PR #242 that supports optional on-device wake words, you can install use this config

substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom Echo
  micro_wake_word_model: okay_nabu # hey_jarvis and hey_mycroft are also supported
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://HarvsG/firmware/wake-word-voice-assistant/m5stack-atom-echo.yaml@patch-3
esphome:
  name: ${name}
  name_add_mac_suffix: True
  friendly_name: ${friendly_name}
api:
  encryption:
    key: SOMEAPIKEY
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Enjoy!

Agree. Bit disappointed in the developers response to this issue. This solution worked for me. Thanks

austingreisman commented 2 days ago

I've also gotten this problem

nem0nic commented 1 day ago

Same issue since the previous update. Skipped the last update because I didn't have time to be screwing around with it hoping the latest version would have the fix but same issue with the latest version as well.

HAOS on Rpi5 8GB

ERROR Error binary size: Error: The OTA partition on the ESP is too small. ESPHome needs to resize this partition, please flash over USB

freeDom- commented 1 day ago

Actually it looks like there's unutilized space between end of phy-init (0xc000) to start of ota-0 (0x10000). That seems to be an extra 16kb we can use for our app.

Thanks for your investigation. Are you willing to open a PR for this?

nem0nic commented 1 day ago

It'd already part of https://github.com/esphome/firmware/pull/242 I believe

HarvsG commented 1 day ago

It'd already part of https://github.com/esphome/firmware/pull/242 I believe

Nope #242 is just a different firmware. I had to remove improv ble to make the space.

freeDom- commented 1 day ago

I think rolling back the changes from @jesserockz is not a real solution, nor removing other features. If NVS is too big and there is unused space left, redistributing space / resizing partitions seems to be the better option!

almoney commented 10 hours ago

Does anyone know if any developer is looking into this issue yet? Seems like it's wide spread and still occurring. As mentioned in another thread, I didn't see anything that appeared to address this in the latest esphome release (2024.7.1).

No, it doesn't look as if it has got any traction In the meantime, I've opened a PR #242 that supports optional on-device wake words, you can install use this config

substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom Echo
  micro_wake_word_model: okay_nabu # hey_jarvis and hey_mycroft are also supported
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://HarvsG/firmware/wake-word-voice-assistant/m5stack-atom-echo.yaml@patch-3
esphome:
  name: ${name}
  name_add_mac_suffix: True
  friendly_name: ${friendly_name}
api:
  encryption:
    key: SOMEAPIKEY
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Enjoy!

Agree. Bit disappointed in the developers response to this issue. This solution worked for me. Thanks

I really appreciate this config help! I am up and running with the local wake word! Thank you!

jarrah31 commented 4 hours ago 
substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom Echo
  micro_wake_word_model: okay_nabu # hey_jarvis and hey_mycroft are also supported
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://HarvsG/firmware/wake-word-voice-assistant/m5stack-atom-echo.yaml@patch-3
esphome:
  name: ${name}
  name_add_mac_suffix: True
  friendly_name: ${friendly_name}
api:
  encryption:
    key: SOMEAPIKEY
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Enjoy!

Is there any way to fully reset/wipe the Atom Echo because I'm unfortunately still ending up with a boot loop when I install this config via my ESPHome dashboard USB method.

[20:22:26]E (823) esp_image: Image length 1835616 doesn't fit in partition length 1835008
[20:22:26]E (823) boot: OTA app partition slot 0 is not bootable
[20:22:26]E (825) esp_image: image at 0x1d0000 has invalid magic byte (nothing flashed here?)
[20:22:26]E (833) boot: OTA app partition slot 1 is not bootable
[20:22:26]E (839) boot: No bootable app partitions in the partition table
nem0nic commented 4 hours ago 
substitutions:
  name: m5stack-atom-echo
  friendly_name: M5Stack Atom Echo
  micro_wake_word_model: okay_nabu # hey_jarvis and hey_mycroft are also supported
packages:
  m5stack.atom-echo-wake-word-voice-assistant: github://HarvsG/firmware/wake-word-voice-assistant/m5stack-atom-echo.yaml@patch-3
esphome:
  name: ${name}
  name_add_mac_suffix: True
  friendly_name: ${friendly_name}
api:
  encryption:
    key: SOMEAPIKEY
wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

Enjoy!

Is there any way to fully reset/wipe the Atom Echo because I'm unfortunately still ending up with a boot loop when I install this config via my ESPHome dashboard USB method.

Go into ESPHome and find your device and hit the 3 dots on the right on the device's card and select "Clean Build Files" and try the install again.

jarrah31 commented 2 hours ago

Go into ESPHome and find your device and hit the 3 dots on the right on the device's card and select "Clean Build Files" and try the install again.

ahh thank you so much, that did the trick!