idodov commented 4 days ago

Bug report

Home Assistant Entities Unavailable After ESPHome 2024.10.0 Installation

After updating to the new ESPHome 2024.10 version, the Ulanzi device disappeared from Home Assistant. Only a rolling back to version 2024.9.2 resolved the issue.

While the device remains accessible via the local control URL after the 2024.10.0 installation, its services and entities are unavailable in Home Assistant.

Additional information

used EspHoMaTriXv2 Firmware:
- 2023.9.1

alting commented 4 days ago

Confirm

clfberlin commented 4 days ago

I too have multiple challenges with 2024.10.0 and my Ulanzis. What I do see is that devices with the newer version seem to work.

See: https://github.com/lubeda/EspHoMaTriXv2/releases/tag/2024.3.0

external_components:
  - source:
      type: git
      url: https://github.com/lubeda/EspHoMaTriXv2
      ref: 2024.3.0
    refresh: 120m 
    components: [ehmtxv2]

(The "120m" caused an error. I changed it to 120min and it installed, don't know if that is correct, though).

But I have additional issues with the integration. It keeps finding my Ulanzis, eventhough they are already configured.

And one really annoying one: My Ulanzis randomly switch identities! But I'll open a separte one for that,

idodov commented 4 days ago

@clfberlin I attempted to work around the issue by updating to version 2024.3.0 (EspHome 2024.10.0), but unfortunately, it did not resolve the problem. I had to flash back to version 2023.9.1 (EspHome 2024.9.2) and rejoin the Ulanzi device to Home Assistant. By doing that it also changed the swich entity name for me.

BTW the refresh value is in seconds: refresh: 120s not minutes.

clfberlin commented 4 days ago

Thanks! I have three Ulanzis in use. Let's call them Ulanzi01, Ulanzi02 and Ulanzi03.

phenomenon 1: If I get Ulanzi01 running, it tests out fine. I then update Ulanzi03 (or all three) and suddenly Ulanzi03 is addressed as Ulanzi01 oder Ulanzi02. They just randomly switch identities
phenomenon 2: My Ulanzi integrations are not properly found. I.e. Ulanzi01 is found in integrations. It discoverso Ulanzi02. But when I want to add it, it is 'already configured' (without actually showing up).
There seems to be an issue with MAC addresses (which might be the reason, why all Ulanzis are confused). The one Ulanzi that does appear as an integration has "MAC: FC:3F:00:00:00:00"

All three Ulanzis can be reached via browser. And they react ok there. When I push the left button on Ulanzi02, the state on the page changes accordingly for Ulanzi02.

But when I try to add an Ulanzi that was discovered as an integration, I get the following warnings:

[component:170] | Component api cleared Warning flag
[esp-idf:000] | E (320493) system_api: Base MAC address from BLK3 of EFUSE version error, version = 0
[component:157] | Component api set Warning flag: unspecified

clfberlin commented 4 days ago

Oh, and another one: In the integrations only one Ulanzi appears. In the overview it is Ulanzi02. But if click on 'device', it has the name 'Ulanzi01' in the top left corner. This goes hand in hand with the mix up when I add a discovered Ulanzi-integration. From that point on the added Ulanzi02 reacts when I address Ulanzi01.

kaufe commented 4 days ago

Hi, can confirm the same Problem here, used also a Ulanzi Device.

external_components:
  - source:
      type: git
      url: https://github.com/lubeda/EspHoMaTriXv2
      ref: 2024.5.1
    refresh: 120s
    components: [ehmtxv2]

tried ref: main, 2024.3.0, 2024.4.1 and 2024.5.1 on each of them i got the error:

ehmtxv2: [source /config/esphome/ke-kueche-esphome-ulanzi.yaml:267]
  id: rgb8x32
  icons2html: True
  matrix_component: ehmtx_display
  time_component: ehmtx_time

  [show_date] is an invalid option for [ehmtxv2]. Did you mean [show_dow]?
  show_date: False
  time_format: %H:%M
  date_format: %d.%m.
.....

He compiles, when i remove the Line "show_date: False", but after the reboot i get a black screen and he reboot each few minutes and in Home Assistant i get always a new device.

<!--StartFragment-->
09:54:00 | [E] | [api:129] | No client connected to API. Rebooting...
10:09:08 | [E] | [api:129] | No client connected to API. Rebooting...
<!--EndFragment-->

trip5 commented 4 days ago

@clfberlin This sounds to me like it could be an mDNS issue or an issue with the ESPHome / Home Assistant discovery process. It's certainly not directly an eHMTX problem. If this error is due to mDNS, it's going to have this kind of error even if your YAML is barebones.

Why not try a more thorough "cleaning-out" by 1) unplugging ALL clocks, 2) deleting ALL eHMTXv2 clocks from the ESPHome integration (and any entities leftover - double check, not just in the ESPHome integration), 3) reboot your router by unplugging it for a full minute (if it's an mDNS issue this will help), 4) flash one manually via USB (if the integration shows up in HA, nevermind it until flashing is done), 5) when it comes back online, add that one to HA properly... repeat steps 4 and 5 until finished.

I did have a similar problem with mine because I decided to switch names of my clocks and there was some leftover integrations and entities in HA... until I did the above cleaning.

Also, you really should open your own issue instead of hijacking another...

As for the original issue, that sounds very concerning. Got me a bit worried to upgrade my ESPHome container. Have y'all tested this out with other devices...? Does it just affect the EHMTXv2s?

lubeda commented 4 days ago

I've got the same problem, but no idea yet where it comes from. I have to check the esphome changelog for hints

clfberlin commented 4 days ago

@trip5 - Thanks. I already removed the Ulanzis from integration and from ESPHome and reinstalled via USB. That didn't do the job. But what I didn't do was sweep through the system to delete all entities and such. Should be worth a try. And yes, I also considered this to be an ESPHome problem (and it might be). But I do have other devices in there and unlike the Ulanzis, they all have proper MAC-addresses. To me this indicates a problem that ESPHome 2024.10.0 might specifically cause for ESPHoMaTriXv2 (or the Ulanzis in this case).

lubeda commented 4 days ago

possible troublemakers from the esphome change-log:

[wifi] Use custom MAC address if programmed esphome#7498 by @kbx81

[wifi] Fix error message when no custom MAC is set esphome#7515 by @kbx81

[wifi] Replace USE_ESP32_IGNORE_EFUSE_MAC_CRC with IDF’s CONFIG_ESP_MAC_IGNORE_MAC_CRC_ERROR esphome#7502 by @kbx81

Perhaps also the changes to the idf

idodov commented 4 days ago

@clfberlin To resolve the issue, you need the ESPHome 2024.9.2 backup and should downgrade the ESPHome version from 2024.10.0 to 2024.9.2.

Perform a full restore of Home Assistant due to the deletion of Ulanzi YAML files. After the full restore, the clocks will still be unavailable because their firmware is 2024.10.0.
You will need to flash them again with 2024.9.3. On the ESPHome add-on page, create a firmware file for each clock (from Plug into this computer option), then restore them using the ESPHome console https://web.esphome.io/?dashboard_install when the clock is connected via USB to the computer. Home Assistant will then rediscover them.
After this, you can upgrade the ESPHome version again to 2024.10.0, but do not flash it to the Ulanzi clocks until a proper fix is released.

lubeda commented 4 days ago

Downgrade is no option, i think we should watch this: https://github.com/esphome/issues/issues/6333

clfberlin commented 4 days ago

@idodov Thanks for the input. You are talking about ESPHome downgrade only, right? Because just today I had a major cleanup session in my system, getting rid of many, many obsolete entities and old configuration stuff. ZHA alone involves 100 devices in my household. A full restore of the entire system would be a setback of many clean-up hours. I'd rather sit out the situation and take the Ulanzis out of the game for a while until a proper solution is at hand. I of course could use a second (backup) system, install the older backup there, flash the Ulanzis and then wait. I would consider that. But the situation is rather fresh and as @lubeda points out, there seem to be other issues as well. Thanks and cheers!

clfberlin commented 3 days ago

Downgrade is no option, i think we should watch this: esphome/issues#6333

Yes, that looks promissing.

idodov commented 3 days ago

@lubeda You have any clue what exactly needs to be changed in the YAML file? It's definitely not this: (or other combinations I tried)

esp32:
  board: esp32dev
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_FREERTOS_UNICORE: y
    advanced:
      ignore_efuse_custom_mac: true
      ignore_efuse_mac_crc: true

got error:

Failed config

light.neopixelbus: [source /config/esphome/pixel.yaml:233]

  This feature is only available with frameworks ['arduino'].
  platform: neopixelbus
  id: ehmtx_light
  type: GRB
  internal: True
  variant: WS2812
  pin: GPIO32
  num_leds: 256
  color_correct: 
    - 30%
    - 30%
    - 30%
  gamma_correct: 2.0

clfberlin commented 3 days ago

Hm. Isn't EspHoMaTriX based on Arduino (rather than ESP-IDF)?

idodov commented 2 days ago

Hm. Isn't EspHoMaTriX based on Arduino (rather than ESP-IDF)?

These options can't be configured for the Arduino framework type, making the current "fix" ineffective for our situation.

lubeda commented 2 days ago

Hi, Arduino vs. ESP-IDF is more a internal compiler/library thing. So, i use these options:

esp32:
  board: "$board"
  framework:
    type: esp-idf
    advanced:
        ignore_efuse_custom_mac: true
        ignore_efuse_mac_crc: true

external_components:
  - source:
      type: git
      url: https://github.com/lubeda/EspHoMaTriXv2
      ref: latest
    refresh: 60s 
    components: [ ehmtxv2 ]

With this i have got a working ulanzi.

bruberg commented 2 days ago

Using the neopixelbus platform (as demonstrated in the EspHoMaTriXv2 documentation) will not work with esp-idf. Any suggestion to work around that?

light.neopixelbus: [source /config/esphome/ulanzi.yaml:385]

  This feature is only available with frameworks ['arduino'].
  platform: neopixelbus
  id: ehmtx_light
  type: GRB
  variant: WS2812
  pin: GPIO32
  num_leds: 256
  color_correct: 
    - 30%
    - 30%
    - 30%
  gamma_correct: 2.0
  name: ulanzi Light

clfberlin commented 2 days ago

Yes, same error here. Failed config.

idodov commented 2 days ago

@lubeda How should we address the neopixelbus error?

lubeda commented 2 days ago

The neopixel lib only works with arduino I use:

light:
  - platform: esp32_rmt_led_strip
    rmt_channel: 0
    id: ehmtx_light
    rgb_order: GRB
    chipset: WS2812
    pin: $matrix_pin
    num_leds: 256
    color_correct: [40%, 40%, 40%]
    name: "$devicename Light"
    max_refresh_rate: $timing
    restore_mode: ALWAYS_OFF

This is only a work around the troublemaker is still in the esp-idf

clfberlin commented 2 days ago

Thanks for the workaround. What do you set as max_refresh_rate? A definition for $timing cannot be found.

lafriks commented 1 day ago

Thanks for the workaround. What do you set as max_refresh_rate? A definition for $timing cannot be found.

works for me with max_refresh_rate: 16ms

rrozema commented 17 hours ago

I intended to -very proudly- show off my work around I found for all of the same issues I had with my ulanzi TC001. Only to find that you, @lubeda, beat me to exactly the same workaround with 2 days :-). Thanks anyway!

lubeda / EspHoMaTriXv2

[BUG] ESPHoMaTriXv2 not compatible with ESPHome 2024.10.0 #248

Bug report

Home Assistant Entities Unavailable After ESPHome 2024.10.0 Installation

Additional information