search

trvrnrth / esphome-bsec-bme680

ESPHome component for the Bosch BME680 sensor via BSEC providing temperature, humidity, pressure and indoor air quality measurements.
85 stars 17 forks source link

Intermittent ESP32 Crashes #21

Closed sjtrny closed 3 years ago

sjtrny commented 3 years ago

Hi there, I'm trying to resolve an intermittent crash/reboot issue involving this component.

To me it looks like something is going wrong when reading data from the device. I should also point out that in the logs it frequently mentions that:

[13:49:07][V][app:081]: A component took a long time in a loop() cycle (0.62 s).

Traceback

[13:49:10]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[13:49:10]Core 1 register dump:
[13:49:10]PC      : 0x400e9784  PS      : 0x00060e30  A0      : 0x800e981a  A1      : 0x3ffb1920  
INFO Need to fetch platformio IDE-data, please stand by
INFO Running:  platformio run -d co2 -t idedata
WARNING Decoded 0x400e9784: AsyncEventSourceClient::_runQueue() at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
[13:49:13]A2      : 0x3ffd6a0c  A3      : 0x3ffb9a18  A4      : 0x08000000  A5      : 0x3ffb946e  
[13:49:13]A6      : 0x00000a0d  A7      : 0x0a0d6d30  A8      : 0x800e9790  A9      : 0x3ffb1900  
[13:49:13]A10     : 0x00000001  A11     : 0x3ffd6ed4  A12     : 0xebc0049c  A13     : 0x3ffb9877  
[13:49:13]A14     : 0x00000000  A15     : 0x0000006d  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[13:49:13]EXCVADDR: 0x00000009  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  
[13:49:13]
[13:49:13]Backtrace: 0x400e9784:0x3ffb1920 0x400e9817:0x3ffb1940 0x400e9833:0x3ffb1960 0x400e9889:0x3ffb1980 0x400dd1c2:0x3ffb19b0 0x400da211:0x3ffb19d0 0x400da361:0x3ffb1a00 0x400e3905:0x3ffb1a20 0x400e3932:0x3ffb1a50 0x400d950d:0x3ffb1ab0 0x400d9581:0x3ffb1af0 0x400d959d:0x3ffb1b10 0x400d9003:0x3ffb1b30 0x400ef4c5:0x3ffb1b60 0x400efc6d:0x3ffb1b80 0x400ef016:0x3ffb1c50 0x400ef361:0x3ffb1df0 0x400d8c86:0x3ffb1ee0 0x401715f5:0x3ffb1f20 0x401716c1:0x3ffb1f40 0x400e1e7d:0x3ffb1f60 0x400e456e:0x3ffb1f90 0x400f380d:0x3ffb1fb0 0x40089aa1:0x3ffb1fd0
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x400e9784: AsyncEventSourceClient::_runQueue() at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400e9817: AsyncEventSourceClient::_queueMessage(AsyncEventSourceMessage*) at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400e9833: AsyncEventSourceClient::write(char const*, unsigned int) at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400e9889: AsyncEventSource::send(char const*, char const*, unsigned int, unsigned int) at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400dd1c2: std::_Function_handler<void (int, char const*, char const*), esphome::web_server::WebServer::setup()::{lambda(int, char const*, char const*)#2}>::_M_invoke(std::_Any_data const&, int&&, char const*&&, int&&) at /Users/steve/Desktop/co2/co2/src/esphome/components/web_server/web_server.cpp:63
 (inlined by) _M_invoke at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1871
WARNING Decoded 0x400da211: std::function<void (int, char const*, char const*)>::operator()(int, char const*, char const*) const at /Users/steve/Desktop/co2/co2/src/esphome/components/logger/logger.cpp:171
 (inlined by) esphome::CallbackManager<void (int, char const*, char const*)>::call(int, char const*, char const*) at /Users/steve/Desktop/co2/co2/src/esphome/core/helpers.h:209
 (inlined by) esphome::logger::Logger::log_message_(int, char const*, int) at /Users/steve/Desktop/co2/co2/src/esphome/components/logger/logger.cpp:113
WARNING Decoded 0x400da361: esphome::logger::Logger::log_vprintf_(int, char const*, int, char const*, __va_list_tag) at /Users/steve/Desktop/co2/co2/src/esphome/components/logger/logger.cpp:171
WARNING Decoded 0x400e3905: esphome::esp_log_vprintf_(int, char const*, int, char const*, __va_list_tag) at /Users/steve/Desktop/co2/co2/src/esphome/core/log.cpp:32
WARNING Decoded 0x400e3932: esphome::esp_log_printf_(int, char const*, int, char const*, ...) at /Users/steve/Desktop/co2/co2/src/esphome/core/log.cpp:14
WARNING Decoded 0x400d950d: esphome::i2c::I2CComponent::raw_receive(unsigned char, unsigned char*, unsigned char) at /Users/steve/Desktop/co2/co2/src/esphome/components/i2c/i2c.cpp:204
WARNING Decoded 0x400d9581: esphome::i2c::I2CComponent::read_bytes(unsigned char, unsigned char, unsigned char*, unsigned char, unsigned int) at /Users/steve/Desktop/co2/co2/src/esphome/components/i2c/i2c.cpp:204
WARNING Decoded 0x400d959d: esphome::i2c::I2CDevice::read_bytes(unsigned char, unsigned char*, unsigned char, unsigned int) at /Users/steve/Desktop/co2/co2/src/esphome/components/i2c/i2c.cpp:204
WARNING Decoded 0x400d9003: esphome::bme680_bsec::BME680BSECComponent::read_bytes_wrapper(unsigned char, unsigned char, unsigned char*, unsigned short) at /Users/steve/Desktop/co2/co2/src/esphome/components/bme680_bsec/bme680_bsec.cpp:145
WARNING Decoded 0x400ef4c5: bme680_get_regs at /Users/steve/Desktop/co2/co2/.piolibdeps/co2/BSEC Software Library/src/bme680/bme680.c:657
WARNING Decoded 0x400efc6d: read_field_data at /Users/steve/Desktop/co2/co2/.piolibdeps/co2/BSEC Software Library/src/bme680/bme680.c:1215
 (inlined by) bme680_get_sensor_data at /Users/steve/Desktop/co2/co2/.piolibdeps/co2/BSEC Software Library/src/bme680/bme680.c:704
WARNING Decoded 0x400ef016: Bsec::readProcessData(long long, bsec_bme_settings_t) at /Users/steve/Desktop/co2/co2/.piolibdeps/co2/BSEC Software Library/src/bsec.cpp:325
WARNING Decoded 0x400ef361: Bsec::run(long long) at /Users/steve/Desktop/co2/co2/.piolibdeps/co2/BSEC Software Library/src/bsec.cpp:242
WARNING Decoded 0x400d8c86: esphome::bme680_bsec::BME680BSECComponent::loop() at /Users/steve/Desktop/co2/co2/src/esphome/components/bme680_bsec/bme680_bsec.cpp:76 (discriminator 1)
WARNING Decoded 0x401715f5: esphome::Component::call_loop() at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1754
WARNING Decoded 0x401716c1: esphome::Component::call() at /Users/steve/Desktop/co2/co2/src/esphome/core/component.cpp:82
WARNING Decoded 0x400e1e7d: esphome::Application::loop() at /Users/steve/Desktop/co2/co2/src/esphome/core/application.cpp:72
WARNING Decoded 0x400e456e: loop() at /Users/steve/Desktop/co2/co2/src/main.cpp:381
WARNING Decoded 0x400f380d: loopTask(void*) at /Users/steve/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:19
WARNING Decoded 0x40089aa1: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)
[13:49:14]
[13:49:14]Rebooting...

Config

esphome:
  name: co2
  platform: ESP32
  board: esp32doit-devkit-v1

# Enable logging
logger:
  level: VERY_VERBOSE

status_led:
  pin:
    number: GPIO2

wifi:
  ssid: "BLAH BLAH"
  password: "BLAH BLAH"
  power_save_mode: none
  fast_connect: True

web_server:
  port: 80

# Enable Home Assistant API
api:

ota:

time:
  - platform: homeassistant
    id: homeassistant_time

binary_sensor:
  - platform: status
    name: "co2 Status"

i2c:
  sda: 13 # D3 on breakout
  scl: 14 # 13 on breakout
  scan: True
  id: bus_a

bme680_bsec:
    # i2c address
    # -----------
    # Common values are:
    # - 0x76
    # - 0x77
    # Default: 0x76
    address: 0x76

sensor:
  - platform: uptime
    name: "BME680 Uptime Sensor"

  - platform: bme680_bsec
    temperature:
      # Temperature in °C
      name: "BME680 Temperature"
      filters:
        - median
    pressure:
      # Pressure in hPa
      name: "BME680 Pressure"
      filters:
        - median
    humidity:
      # Relative humidity %
      name: "BME680 Humidity"
      filters:
        - median
    gas_resistance:
      # Gas resistance in Ω
      name: "BME680 Gas Resistance"
      filters:
        - median
    iaq:
      # Indoor air quality value
      name: "BME680 IAQ"
      filters:
        - median
    co2_equivalent:
      # CO2 equivalent estimate in ppm
      name: "BME680 CO2 Equivalent"
      filters:
        - median
    breath_voc_equivalent:
      # Volatile organic compounds equivalent estimate in ppm
      name: "BME680 Breath VOC Equivalent"
      filters:
        - median

text_sensor:
  - platform: bme680_bsec
    iaq_accuracy:
      # IAQ accuracy as a text value of Stabilizing, Uncertain, Calibrating, Calibrated
      name: "BME680 IAQ Accuracy"
trvrnrth commented 3 years ago

That looks a little bit like it might be an OOM crash resulting from excessive network logging. Ask a silly question perhaps but do you still see crashes running at say INFO log level?

sjtrny commented 3 years ago

I'll leave it running overnight and see what happens with just INFO log level

sjtrny commented 3 years ago

1 hour after start it crashed with logging set to INFO level

# Enable logging
logger:
  level: INFO
[02:03:56][I][logger:166]: Log initialized
[02:03:56][I][app:029]: Running through setup()...
[02:03:56][I][wifi:194]: WiFi Connecting to '15 Rochdale'...
[02:03:59][I][wifi:457]: WiFi Connected!
[02:03:59][I][app:059]: setup() finished successfully!
[02:03:59][I][app:105]: ESPHome version 1.15.3 compiled on Jan  3 2021, 02:02:37
[02:03:59][I][i2c:033]: Scanning i2c bus for active devices...
[02:03:59][I][i2c:040]: Found i2c device at address 0x76
[02:05:56][I][ota:046]: Boot seems successful, resetting boot loop counter.
[02:51:34]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[02:51:34]Core 1 register dump:
[02:51:34]PC      : 0x400e6194  PS      : 0x00060930  A0      : 0x800e622a  A1      : 0x3ffb1cf0  
INFO Need to fetch platformio IDE-data, please stand by
INFO Running:  platformio run -d co2 -t idedata
WARNING Decoded 0x400e6194: AsyncEventSourceClient::_runQueue() at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
[02:51:35]A2      : 0x3ffb9878  A3      : 0x5e43420f  A4      : 0x00000000  A5      : 0x3ffb2cf8  
[02:51:35]A6      : 0x0a0d0a0d  A7      : 0x7d323935  A8      : 0x00000000  A9      : 0x3ffb1cd0  
[02:51:35]A10     : 0x00000000  A11     : 0x00000054  A12     : 0x00000054  A13     : 0x3ffb2664  
[02:51:35]A14     : 0x00000000  A15     : 0x000a0d0a  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[02:51:35]EXCVADDR: 0x00000008  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  
[02:51:35]
[02:51:35]Backtrace: 0x400e6194:0x3ffb1cf0 0x400e6227:0x3ffb1d10 0x400e6243:0x3ffb1d30 0x400e6299:0x3ffb1d50 0x400db519:0x3ffb1d80 0x4016d999:0x3ffb1dc0 0x400da291:0x3ffb1de0 0x400da2b7:0x3ffb1e10 0x400d907a:0x3ffb1e40 0x400d90a2:0x3ffb1e60 0x400da304:0x3ffb1e90 0x400d6ce1:0x3ffb1ec0 0x400d70af:0x3ffb1ee0 0x4016d831:0x3ffb1f20 0x4016d8fd:0x3ffb1f40 0x400ded82:0x3ffb1f60 0x400e1262:0x3ffb1f90 0x400efbb1:0x3ffb1fb0 0x400896ad:0x3ffb1fd0
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x400e6194: AsyncEventSourceClient::_runQueue() at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400e6227: AsyncEventSourceClient::_queueMessage(AsyncEventSourceMessage*) at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400e6243: AsyncEventSourceClient::write(char const*, unsigned int) at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400e6299: AsyncEventSource::send(char const*, char const*, unsigned int, unsigned int) at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1858
WARNING Decoded 0x400db519: esphome::web_server::WebServer::on_sensor_update(esphome::sensor::Sensor*, float) at /Users/steve/Desktop/co2/co2/src/esphome/components/web_server/web_server.cpp:63
WARNING Decoded 0x4016d999: std::_Function_handler<void (float), esphome::Controller::setup_controller()::{lambda(float)#2}>::_M_invoke(std::_Any_data const&, float&&) at /Users/steve/Desktop/co2/co2/src/esphome/core/controller.cpp:29
 (inlined by) _M_invoke at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1871
WARNING Decoded 0x400da291: std::function<void (float)>::operator()(float) const at /Users/steve/Desktop/co2/co2/src/esphome/components/sensor/sensor.cpp:85
WARNING Decoded 0x400da2b7: esphome::CallbackManager<void (float)>::call(float) at /Users/steve/Desktop/co2/co2/src/esphome/components/sensor/sensor.cpp:85
 (inlined by) esphome::sensor::Sensor::internal_send_state_to_frontend(float) at /Users/steve/Desktop/co2/co2/src/esphome/components/sensor/sensor.cpp:93
WARNING Decoded 0x400d907a: esphome::sensor::Filter::output(float) at /Users/steve/Desktop/co2/co2/src/esphome/components/sensor/filter.cpp:83
WARNING Decoded 0x400d90a2: esphome::sensor::Filter::input(float) at /Users/steve/Desktop/co2/co2/src/esphome/components/sensor/filter.cpp:83
WARNING Decoded 0x400da304: esphome::sensor::Sensor::publish_state(float) at /Users/steve/Desktop/co2/co2/src/esphome/components/sensor/sensor.cpp:85
WARNING Decoded 0x400d6ce1: esphome::bme680_bsec::BME680BSECComponent::publish_state_(esphome::sensor::Sensor*, float) at /Users/steve/Desktop/co2/co2/src/esphome/components/bme680_bsec/bme680_bsec.cpp:117
WARNING Decoded 0x400d70af: esphome::bme680_bsec::BME680BSECComponent::loop() at /Users/steve/Desktop/co2/co2/src/esphome/components/bme680_bsec/bme680_bsec.cpp:107
WARNING Decoded 0x4016d831: esphome::Component::call_loop() at /Users/steve/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/functional:1754
WARNING Decoded 0x4016d8fd: esphome::Component::call() at /Users/steve/Desktop/co2/co2/src/esphome/core/component.cpp:82
WARNING Decoded 0x400ded82: esphome::Application::loop() at /Users/steve/Desktop/co2/co2/src/esphome/core/application.cpp:72
WARNING Decoded 0x400e1262: loop() at /Users/steve/Desktop/co2/co2/src/main.cpp:381
WARNING Decoded 0x400efbb1: loopTask(void*) at /Users/steve/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:19
WARNING Decoded 0x400896ad: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)
[02:51:35]
[02:51:35]Rebooting...
trvrnrth commented 3 years ago

https://github.com/esphome/issues/issues/1181#issuecomment-622586665 seems like the same thing. It looks like you're using HA anyway so maybe you can live without the webserver?

sjtrny commented 3 years ago

Yes it looks similar. I think it might be an interaction between this component (and those others in the linked issue) and the web server component because it never happens on my other esphome device which only does PWM/switches.

Perhaps something is taking too long to complete or there's not enough resources?

trvrnrth commented 3 years ago

I'm very limited in how much I can yield in the component itself but there is some possibility that the crash is occurring during a state save. I've tidied up/added to the yielding in #24 so I'd be very keen to know if that branch makes any difference for you.

sjtrny commented 3 years ago

Thanks! I've updated my ESP using that branch and I'll report back in a day or so.

sjtrny commented 3 years ago

Those yields seem to have done the trick. I haven’t seen any crashes yet.

trvrnrth commented 3 years ago

Awesome. That was easier than I'd feared then! Thanks for taking the time to test things out :)