esphome / issues

Issue Tracker for ESPHome
https://esphome.io/
292 stars 36 forks source link

Multiple compile retries to successful compile #4409

Closed glbailey closed 9 months ago

glbailey commented 1 year ago

The problem

Attempts to compile and upload for each device takes 8 to 10 retry attempts before succeeding, with seemingly random errors at each retry. No YAML or other changes made at each retry, but eventually compile succeeds. Note this example is 1 of 12 different devices being used in this system including multiple Sonoff S31 devices, ESP8266 D1 mini, eSP32Cam and ESP32dev devices using esp-idf and arduino platforms. Compilation failures affect ALL of these devices and platforms. No obvious device dependencies. Restarting of ESPHome add-on sometimes shortens the number of retries before success and a full reboot of Home Assistant often allows first time compile success for 3 to 4 devices before returning to multiple failed compilations. This issue is not new; has been occurring since Home Assistant versions back to at least October 2022. I have increased cores from 2 to 4 and increased memory from 3GB to 6GB with no change in (seemingly) random compile errors. Using "clean build files" before a retry seems to make no difference. Attached are results from a series of 8 retries before we got a successful compile and upload showing the errors at each step.

Which version of ESPHome has the issue?

2023.3.2

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

2023.3.5

What platform are you using?

ESP32-IDF

Board

ESP32 DEVKIT V1

Component causing the issue

compiler

Example YAML snippet

esphome:
  name: hot-tub-monitor

esp32:
  board: esp32dev
  framework:
    type: esp-idf

esp32_ble_tracker:
  scan_parameters:
    interval: 1100ms
    window: 1100ms
    active: true

bluetooth_proxy:
  active: true

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: "REDACTED"

ota:
  password: "REDACTED"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Hot-Tub-Monitor Fallback Hotspot"
    password: "rBXtp691tFBz"

# captive_portal: - not available in esp-idf

# Hot tub monitor sensor array - temperature for room and water, room light level,
# TDS and pH sensors with 128x64 OLED display

# DS18B20 Temperature Sensor - default 60 sec update interval
dallas:
  - pin: GPIO2
    update_interval: 15s

font:
# gfonts://family[@weight]
  - file: "gfonts://Roboto"
    id: roboto
    size: 14
  - file: 'slkscr.ttf'
    id: font1
    size: 12
  - file: 'BebasNeue-Regular.ttf'
    id: font2
    size: 48
  - file: 'arial.ttf'
    id: font3
    size: 14

# ---------------------
# SSD1306 Display Module
# Vcc 3.3V to 5V
i2c:
  sda: GPIO21
  scl: GPIO22

display:
  - platform: ssd1306_i2c
    model: "SSD1306 128x64"
#    reset_pin: GPIO4
#    address: 0x3C    # marked as 0x78 (shifted 1 bit left for r/w bit)
#    lambda: |-
#      it.printf(64, 0, id(roboto), TextAlign::TOP_CENTER, "Ta:%.1f°C Tw:%.1f°C", id(sunroom_temperature).state, id(hot_tub_temperature).state);
#      it.printf(0, 60, id(font2), TextAlign::BASELINE_LEFT, "%.1f", id(hot_tub_water_ph).state);
#      it.print(65, 60, id(font3), TextAlign::BASELINE_RIGHT, "pH");
#      it.printf(127, 23, id(font3), TextAlign::TOP_RIGHT, "%.0fppm", id(hot_tub_water_tds).state);
#      it.printf(127, 60, id(font3), TextAlign::BASELINE_RIGHT, "%.3fV", id(voltage_ph_sensor).state);
    address: 0x3C    # marked as 0x78 (shifted 1 bit left for r/w bit)
    id: hot_tub_display
    contrast: 50%
    pages: # alternate pages to reduce screen burn-in
      - id: page1
        lambda: |-
          it.printf(0, 0, id(roboto), TextAlign::TOP_LEFT, "Ta:%.1f°C Tw:%.1f°C", id(sunroom_temperature).state, id(hot_tub_temperature).state);
          it.printf(0, 60, id(font2), TextAlign::BASELINE_LEFT, "%.1f", id(hot_tub_water_ph).state);
          it.print(65, 60, id(font3), TextAlign::BASELINE_RIGHT, "pH");
          it.printf(127, 23, id(font3), TextAlign::TOP_RIGHT, "%.0fppm", id(hot_tub_water_tds).state);
          it.printf(127, 60, id(font3), TextAlign::BASELINE_RIGHT, "%.3fV", id(voltage_ph_sensor).state);
      - id: page2
        lambda: |-
          it.printf(128, 0, id(roboto), TextAlign::TOP_RIGHT, "Ta:%.1f°C Tw:%.1f°C", id(sunroom_temperature).state, id(hot_tub_temperature).state);
          it.printf(65, 60, id(font2), TextAlign::BASELINE_LEFT, "%.1f", id(hot_tub_water_ph).state);
          it.print(127, 60, id(font3), TextAlign::BASELINE_RIGHT, "pH");
          it.printf(60, 23, id(font3), TextAlign::TOP_RIGHT, "%.0fppm", id(hot_tub_water_tds).state);
          it.printf(60, 60, id(font3), TextAlign::BASELINE_RIGHT, "%.3fV", id(voltage_ph_sensor).state);

interval:
  - interval: 10s
    then:
      - display.page.show_next: hot_tub_display
      - component.update: hot_tub_display

# ADC
sensor:
  # measure pH probe actual voltage for display (calibration aid)
  - platform: adc      # pH Sensor - Vcc=5V Vo=5V*(3/5 attenuator)
    id: voltage_ph_sensor
    name: "VpH Sensor"
    pin: GPIO34
    attenuation: 11db
    update_interval: 5s
    state_class: "measurement"
    accuracy_decimals: 3
    unit_of_measurement: "V"
    filters:
      -  lambda: return x; 

  # measure pH probe and apply calibration factors
  - platform: adc      # measure same input but apply different calibration curve
    id: hot_tub_water_ph
    name: "Hot Tub Water pH"
    pin: GPIO36
    attenuation: 11db
    update_interval: 5s
    state_class: "measurement"
    device_class: ""    # allow pH unit of measure without HA expecting V or mV units
    accuracy_decimals: 3
    unit_of_measurement: "pH"
    # temperature compensate calibrations to 40C from 25C
    # 4.01 buffer solution at 25C = 4.03 pH at 40C
    # 6.86 buffer solution at 25C = 6.84 pH at 40C
    # 9.18 buffer solution at 25C = 9.07 pH at 40C
    # initial bench cal 0.349V for 4.03pH, 1.223V for 6.84pH, 1.898V for 9.07pH
    # after install cal 0.404 for 4.03pH, 1.235V for 6.84pH, 1.990 for 9.10pH
    filters:
      - calibrate_linear:
        - 0.404 -> 4.03
        - 1.235-> 6.84
        - 1.990 -> 9.10
      - lambda: |-
          if (x < 0) return 0;
          else return x;

  # measure TDS sensor and apply conductivity and TDS calculation and temp compensation
  - platform: adc      # TDS Sensor - Vcc=5V Vo=2.3V
    pin: GPIO35
    attenuation: 11db
    name: "Hot Tub Water TDS"
    id: hot_tub_water_tds
    update_interval: never
    device_class: "carbon_dioxide" # use ppm as default units
    state_class: "measurement"
    accuracy_decimals: 0
    unit_of_measurement: "ppm"
# only perform TDS measurement & calc at time of temperature measurement to synchronize
# see hot_tub_temperature on_value for trigger
    filters:
# non-linear response curve - 0.77 factor assuming constant 40degC (no temp comp)
#      - lambda: return ((((133.42 * x * x * x) - (255.86 * x * x) + (857.39 * x)) * 0.5) * 0.77);
# add temperature compensation as per standard formula
#      - lambda: return (((133.42 * x * x * x) - (255.86 * x * x) + (857.39 * x)) * 0.5 / (1 + (.02 * (id(hot_tub_temperature).state - 25))));
# better temp compensation to smooth temperature drift effects with modified constants
      - lambda: return (((133.42 * x * x * x) - (255.86 * x * x) + (857.39 * x)) * 0.5 / (.73 + (.038 * (id(hot_tub_temperature).state - 25))));
      - filter_out: nan # remove error messages whebn no return signal received
# smooth out peaks caused by heater coming on/off and temp measurement lagging
      - sliding_window_moving_average:
          window_size: 10

  - platform: dallas
    address: 0x343c01e0768b5028
    name: "Hot Tub Temperature"
    id: hot_tub_temperature
    device_class: "temperature"
    state_class: "measurement"
    filters:
    on_value:
      then:
        - component.update: hot_tub_water_tds    # update tds with current temp

  - platform: dallas
    address: 0x7f3c01e07652ba28
    name: "Sunroom Temperature"
    id: sunroom_temperature
    device_class: "temperature"
    state_class: "measurement"

  - platform: adc
    pin: GPIO33
    attenuation: 11db
    name: "Sunroom Light Level"
    id: sunroom_light_level
    update_interval: 15s
    device_class: "illuminance"
    state_class: "measurement"
    accuracy_decimals: 0
    unit_of_measurement: "lx"
    filters:
#      # measured slope curve
#      - multiply: 50
      - calibrate_linear:
          # Map 0.0 (from sensor) to 0.0 (true value)
          - 0.14 -> 0.0
          - 1.22 -> 170
      - lambda: |-
          if (x < 0) return 0;
          else return x;

  - platform: wifi_signal
    name: "Hot Tub Monitor WiFi Signal"
    update_interval: 15s
    device_class: signal_strength
    state_class: "measurement"

Anything in the logs that might be useful for us?

###############  Attempt / Failure #1:  ###############
(install wirelessly)
(see ESPHome compile failure 1.txt for full log file)
Excerpt from log showing error:
...
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/bootloader_support/src/idf/bootloader_sha.o
/config/esphome/hot-tub-monitor.yaml:188:1: internal compiler error: Segmentation fault
     id: sunroom_temperature
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/bt/controller/esp32/bt.o
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/src/main.o] Error 1
/tmp/ccAGL0at.s: Assembler messages:
/tmp/ccAGL0at.s:46280: Internal error (Segmentation fault).
Please report this bug.
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/asio/asio/asio/src/asio.o] Error 1
========================= [FAILED] Took 39.96 seconds =========================

###############  Attempt / Failure #2:  ###############
(clean build files, install wirelessly)
(see ESPHome compile failure 2.txt for full log file)
Excerpt from log showing error:
...
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/bt/host/bluedroid/stack/gap/gap_api.o
/data/cache/platformio/packages/framework-espidf/components/bt/host/bluedroid/stack/gap/gap_api.c:109:1: internal compiler error: in expand_all_functions, at cgraphunit.c:2249
 }
 ^
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/bt/host/bluedroid/stack/gap/gap_ble.o
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/bt/host/bluedroid/stack/gap/gap_api.o] Error 1
free(): invalid pointer
========================= [FAILED] Took 23.20 seconds =========================

###############  Attempt / Failure #3:  ###############
(press 'retry')
(see ESPHome compile failure 3.txt for full log file)
Excerpt from log showing error:
...
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/src/esphome/components/api/api_pb2_service.o
during IPA pass: fnsummary
src/esphome/components/api/api_frame_helper.cpp: In static member function 'static _ForwardIterator std::__uninitialized_default_n_1<true>::__uninit_default_n(_ForwardIterator, _Size) [with _ForwardIterator = unsigned char*; _Size = unsigned int]':
src/esphome/components/api/api_frame_helper.cpp:1030:1: internal compiler error: in estimate_num_insns, at tree-inline.c:4322
 }  // namespace esphome
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/src/esphome/components/api/api_frame_helper.o] Error 1
src/esphome/components/api/api_connection.cpp:1031:1: internal compiler error: Segmentation fault
 }  // namespace esphome
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/src/esphome/components/api/api_connection.o] Error 1
========================= [FAILED] Took 18.40 seconds =========================

###############  Attempt / Failure #4:  ###############
(press 'retry')
(see ESPHome compile failure 4.txt for full log file)
Excerpt from log showing error:
...
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/libsodium/libsodium/src/libsodium/crypto_pwhash/scryptsalsa208sha256/crypto_scrypt-common.o
/tmp/cctT2kzI.s: Assembler messages:
/tmp/cctT2kzI.s:176: Warning: unrecognized section type
/tmp/cctT2kzI.s:176: Error: junk at end of line, first unrecognized character is `<'
/tmp/cctT2kzI.s:177: Error: expected comma or colon after symbol name; rest of line ignored
/tmp/cctT2kzI.s:177: Error: syntax error
/tmp/cctT2kzI.s:177: Error: junk at end of line, first unrecognized character is `2'
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/libsodium/libsodium/src/libsodium/crypto_pwhash/scryptsalsa208sha256/crypto_scrypt-common.o] Error 1
/tmp/ccjC6Tbx.s: Assembler messages:
/tmp/ccjC6Tbx.s:37676: Internal error (Segmentation fault).
Please report this bug.
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/libsodium/libsodium/src/libsodium/crypto_core/ed25519/ref10/ed25519_ref10.o] Error 1
========================= [FAILED] Took 71.98 seconds =========================

###############  Attempt / Failure #5:  ###############
(press 'retry')
(see ESPHome compile failure 5.txt for full log file)
Excerpt from log showing error:
...
Linking /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.elf
RAM:   [==        ]  16.5% (used 54040 bytes from 327680 bytes)
Flash: [=======   ]  73.1% (used 1341429 bytes from 1835008 bytes)
Building /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.bin
Creating esp32 image...
Successfully created esp32 image.
double free or corruption (out)
Aborted (core dumped)
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.bin] Error 134
========================= [FAILED] Took 67.16 seconds =========================

###############  Attempt / Failure #6:  ###############
(clean build files, install wirelessly)
(see ESPHome compile failure 6.txt for full log file)
Excerpt from log showing error:
...
Compiling /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/src/main.o
/config/esphome/hot-tub-monitor.yaml:188:1: internal compiler error: Segmentation fault
     id: sunroom_temperature
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://gcc.gnu.org/bugs/> for instructions.

###############  Attempt / Failure #7:
(press 'retry')
(see ESPHome compile failure 7.txt for full log file)
Excerpt from log showing error:
...
Linking /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.elf
collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
compilation terminated.
*** [/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.elf] Error 1
========================= [FAILED] Took 131.65 seconds =========================

###############  Attempt / Success #8:  ###############
(press 'retry')
(see ESPHome compile success 8.txt for full log file)
Excerpt from log showing success:
...
Successfully created esp32 image.
esp32_create_combined_bin(["/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.bin"], ["/data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.elf"])
Wrote 0x158e70 bytes to file /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware-factory.bin, ready to flash to offset 0x0
========================= [SUCCESS] Took 33.82 seconds =========================
INFO Successfully compiled program.
INFO Resolving IP address of hot-tub-monitor.local
INFO  -> 192.168.0.176
INFO Uploading /data/hot-tub-monitor/.pioenvs/hot-tub-monitor/firmware.bin (1347184 bytes)
Uploading: [============================================================] 100% Done...

INFO Waiting for result...
INFO OTA successful
INFO Successfully uploaded program.
INFO Starting log output from hot-tub-monitor.local using esphome API

(note: at this point we have a successful compile and upload)

Additional information

8 attempts to compile, 7 failures. NO changes to yaml file between attempts.

System specifics:

Linux Mint 20.3 Cinnamon running VirtualBox 6.1.38_Ubuntu on 8 core i9-9880 16G Ram 512G SSD HA virtual machine assigned 4 processors: core processor usage 2%, supervisor processor usage 0% HA virtual machine assigned 6GB memory - core memory usage 10.5%, supervisor memory usage 11.1%

Home Assistant: Version core-2023.3.5 Installation Type Home Assistant OS Python Version 3.10.10 Operating System Family Linux Operating System Version 5.15.90 CPU Architecture x86_64

ESPHome: version 2023.3.2 Add-on CPU Usage: 0% Add-on RAM Usage: 11.9%

(see hot-tub-monitor.yaml for actual yaml file being compiled for a standard ESP32dev board.)

Note this is 1 of 12 different devices being used in this system including multiple Sonoff S31 devices, ESP8266 D1 mini, eSP32Cam and ESP32dev devices using esp-idf and arduino platforms. Compilation failures affect ALL of these devices. Restarting of ESPHome add-on sometimes shortens the number of retries before success and a full reboot of Home Assistant often allows first time compile success for 3 to 4 devices before returning to multiple failed compilations. This issue is not new; has been occurring since Home Assistant versions back to at least October 2022. I have increased cores from 2 to 4 and increased memory from 3GB to 6GB with no change in (seemingly) random compile errors.

RoboMagus commented 1 year ago

A compiler Segfaulting is often caused by memory issues. Have you tried replicating these issues on another (non virtual) system?

ssieb commented 1 year ago

This is an "internal compiler error", so not the usual out of memory crash. Given the random nature and weird garbage, I would suspect either RAM or disk data corruption.

glbailey commented 1 year ago
Thanks for the input. The odd thing is that after a full reboot, the next few compiles usually run fine. It slowly gets worse and worse. So, to resolve I have removed ESPHome from Home Assistant and instead added it OUTSIDE of the VM in a Docker/Portainer container. I moved all my devices to this instance and after compiling all devices several times have yet to see any failure. That seems to imply something between the Home Assistant Docker and VirtualBox is having trouble managing memory (?) It was convenient to have the ESPHome instance within Home Assistant, but it is nice to not have to fight with the compiler.
RoboMagus commented 1 year ago

In the devcontainer for ESPHome I regularly get these errors thrown at random where it complains about being unable to allocate memory:

In file included from /esphome/.temp/platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/8.4.0/string:40,
                 from /esphome/.temp/platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/8.4.0/stdexcept:39,
                 from /esphome/.temp/platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/8.4.0/array:39,
                 from /esphome/.temp/platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/8.4.0/tuple:39,
                 from /esphome/.temp/platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/8.4.0/functional:54,
                 from src/esphome/core/helpers.h:5,
                 from src/esphome/components/esp32_bt_classic/utils.h:4,
                 from src/esphome/components/esp32_bt_classic/utils.cpp:3:
/esphome/.temp/platformio/packages/toolchain-xtensa-esp32/xtensa-esp32-elf/include/c++/8.4.0/bits/char_traits.h:40:10: fatal error: .piolibdeps/esp32-office-bt-monitor/libsodium/libsodium/src/libsodium/bits/postypes.h: Cannot allocate memory
 #include <bits/postypes.h>      // For streampos

This occurs even though there is enough system memory available. I reccon this is caused by the docker container being a Linux environment, and thus having to run virtualized on my Windows host machine. This is an issue in the virtualization layer, and not something ESPHome can do anything about.

Though not exactly the same as the issue @glbailey describes, I guess the root cause is the same.

u17194907425 commented 1 year ago

Hi. Just want to check in to say that I am seeing the same behavior (random failures and core dumps while compiling ESPHome firmwares, and pressing retry multiple times to make it work).

I am running HA insid a Virtual Box on Win10. I am yet to try solving this on my own...

something between the Home Assistant Docker and VirtualBox is having trouble managing memory (?)

If that's true, then I wonder why the rest of the Home Assistant is running fine.

ssieb commented 1 year ago

Compiling is very stressful on everything. It's a good way to find issues. :grin:

glbailey commented 1 year ago

Yes, I wonder the same thing, but I could not find a solution to fix that directly.

Removing ESPHome from HA and instead running it in a separate Docker container has resulted in perfect compiles every time with no other issues.

It’s a little less convenient for getting update notifications but I have yet to find a downside to running it separately as far as devices are concerned.

It is less secure though as ESPHome doesn’t have a login/password mechanism (that I have run across).

On Jun 30, 2023, at 12:45 AM, u17194907425 @.***> wrote:

Hi. Just want to check in to say that I am seeing the same behavior (random failures and core dumps while compiling ESPHome firmwares, and pressing retry multiple times to make it work).

I am running HA insid a Virtual Box on Win10. I am yet to try solving this on my own...

something between the Home Assistant Docker and VirtualBox is having trouble managing memory (?)

If that's true, then I wonder why the rest of the Home Assistant is running fine.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

RoboMagus commented 1 year ago

If that's true, then I wonder why the rest of the Home Assistant is running fine.

When compiling complete Esphome firmware, there are a lot of files read and written and this would cause a lot of memory access rapid succession. Home Assistant's memory access pattern is much more 'relaxed' in that sense.

RoboMagus commented 1 year ago

It is less secure though as ESPHome doesn’t have a login/password mechanism (that I have run across).

Seeing as you run the Esphome container in docker already, you could set it up behind a reverse proxy with an autentication layer. It is definately not as easy as just exposing Esphome to your network, but the options are there if you are willing to tinker with it.

glbailey commented 1 year ago

set it up behind a reverse proxy with an autentication layer.

Ah, thank you for that! I shall dive down that rabbit hole next :-)

On Jul 1, 2023, at 2:55 AM, RoboMagus @.***> wrote:

It is less secure though as ESPHome doesn’t have a login/password mechanism (that I have run across).

Seeing as you run the Esphome container in docker already, you could set it up behind a reverse proxy with an autentication layer. It is definately not as easy as just exposing Esphome to your network, but the options are there if you are willing to tinker with it.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

u17194907425 commented 1 year ago

Just wanted to check in and confirm that the described behavior still exists. Yesterday I had to retry compilation 6 or 7 times before it was successful.

ESPHome 2023.10.1 Home Assistant 2023.9.2 Supervisor 2023.10.0 Operating System 11.0 VirtualBox 6

glbailey commented 1 year ago

Yes, I gave up on trying to make it work within Home Assistant which is running in virtual box. Instead, I moved ESPHome to a docker container on the main machine (outside of virtual box). Have not had an issue ever since. So I think the issue is memory management related to Virtual Box but I don’t have the tools or knowhow to dig deeper. All the devices work fine in HA with this configuration and whenever I need to work on or update a device I just start up ESPHome in the docker container.

On Oct 23, 2023, at 1:16 AM, u17194907425 @.***> wrote:

Just wanted to check in and confirm that the described behavior still exists. Yesterday I had to retry compilation 6 or 7 times before it was successful.

ESPHome 2023.10.1 Home Assistant 2023.9.2 Supervisor 2023.10.0 Operating System 11.0 VirtualBox 6

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

u17194907425 commented 1 year ago

@glbailey thanks for sharing. BTW since you have mentioned VirtualBox, which is a valid possible root cause of the issue... Are you on version 6 or have you migrated to v7 already? Coz if you are on 6 there is possibility that the issue is gone on VB7

glbailey commented 1 year ago

Well now that’s a very good point - I’m on 6.1 and haven’t checked for updates in a while. Guess I’m going to have to do some testing to find out.

On Oct 23, 2023, at 9:45 AM, u17194907425 @.***> wrote:

@.***(https://github.com/glbailey) thanks for sharing. BTW since you have mentioned VirtualBox, which is a valid possible root cause of the issue... Are you on version 6 or have you migrated to v7 already? Coz if you are on 6 there is possibility that the issue is gone on VB7

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Mikle61 commented 1 year ago

I had similar behavior to ESPhome. In my case, the culprit was my antivirus program. After turning it off, ESPhome operation returned to normal.

u17194907425 commented 9 months ago

Some updade on my observations: The issue with Multiple compile retries to successful compile has completely gone right after I changed RAM on my virtual instanve of VirtualBox from 4 to 8 GB. Now even quite long compilations are stable again. I do not have to press "retry" any more.

mastersf commented 9 months ago

I'm using docker and set shm size to 500mb from 64mb and still needing to do multiple retry for install (compile, and connect) as well as view logs (connect)

ssieb commented 9 months ago

@mastersf you're mentioning multiple things. Only the compiling is relevant here. And actually, I'm going to close this because it's a VirtualBox issue. If you have other issues, come ask on discord.