greghesp / ha-bambulab

A Home Assistant Integration for Bambu Lab Printers
698 stars 59 forks source link

[Bug] Home Assistant 2024.3.1 crashes/is unstable, Integration v2.0.15 and beta; v2.0.16-dev2 - P1S #485

Closed Takst3in closed 2 months ago

Takst3in commented 3 months ago

Describe the bug

When the integration is enabled, my HA crashed/reboots approc atleast once every hour.

Integration set up via cloud, and local-IP. The lines in debug-log/Home Assistant log that contains information in relation to the integration is provided, I hope this points you to fint the root cause of this.

The printer is only a couple of days old, and this behaviour of the integration crashing, have been like this since day one.

To Reproduce

  1. Enable integration
  2. Wait for 5min -> 1hour (approx)
  3. Home Assistant crashes, and reboots

Expected Behaviour

Stable

What device are you using?

P1S

Diagnostic Output

{
  "home_assistant": {
    "installation_type": "Home Assistant OS",
    "version": "2024.3.1",
    "dev": false,
    "hassio": true,
    "virtualenv": false,
    "python_version": "3.12.2",
    "docker": true,
    "arch": "x86_64",
    "timezone": "Europe/Oslo",
    "os_name": "Linux",
    "os_version": "6.6.20-haos",
    "supervisor": "2024.03.0",
    "host_os": "Home Assistant OS 12.1",
    "docker_version": "24.0.7",
    "chassis": "vm",
    "run_as_root": true
  },
  "custom_components": {
    "hacs": {
      "version": "1.34.0",
      "requirements": [
        "aiogithubapi>=22.10.1"
      ]
    },
    "tibber_data": {
      "version": "0.7.2",
      "requirements": [
        "pyTibber>=0.25.2"
      ]
    },
    "bambu_lab": {
      "version": "2.0.15",
      "requirements": []
    },
    "nordpool": {
      "version": "0.0.14",
      "requirements": [
        "nordpool>=0.2",
        "backoff"
      ]
    },
    "tpms_ble": {
      "version": "0.1",
      "requirements": []
    },
    "blitzortung": {
      "version": "1.0.1",
      "requirements": [
        "paho-mqtt>=1.5.0"
      ]
    },
    "google_home": {
      "version": "1.11.0",
      "requirements": [
        "glocaltokens==0.7.0"
      ]
    },
    "easee": {
      "version": "0.9.58",
      "requirements": [
        "pyeasee==0.8.0"
      ]
    },
    "ham_radio_propagation": {
      "version": "1.1.6",
      "requirements": [
        "xmltodict==0.13.0"
      ]
    },
    "tapo_control": {
      "version": "5.4.13",
      "requirements": [
        "pytapo==3.3.19"
      ]
    },
    "feedparser": {
      "version": "0.1.11",
      "requirements": [
        "feedparser==6.0.10"
      ]
    },
    "nodered": {
      "version": "3.1.3",
      "requirements": []
    },
    "tplink_deco": {
      "version": "3.6.0",
      "requirements": [
        "pycryptodome>=3.12.0"
      ]
    },
    "kia_uvo": {
      "version": "2.22.0",
      "requirements": [
        "hyundai_kia_connect_api==3.17.5"
      ]
    },
    "amshan": {
      "version": "2022.10.2",
      "requirements": [
        "amshan[serial]==2.1.1"
      ]
    },
    "localtuya": {
      "version": "5.2.1",
      "requirements": []
    },
    "frigate": {
      "version": "5.0.1",
      "requirements": [
        "pytz==2022.7"
      ]
    },
    "continuously_casting_dashboards": {
      "version": "1.2.6",
      "requirements": [
        "catt==0.12.11"
      ]
    },
    "smartir": {
      "version": "1.17.9",
      "requirements": [
        "aiofiles>=0.6.0"
      ]
    },
    "edgeos": {
      "version": "2.0.31",
      "requirements": [
        "aiohttp"
      ]
    },
    "average": {
      "version": "2.3.3",
      "requirements": []
    }
  },
  "integration_manifest": {
    "domain": "bambu_lab",
    "name": "Bambu Lab",
    "codeowners": [
      "@greghesp",
      "@AdrianGarside"
    ],
    "config_flow": true,
    "dependencies": [
      "device_automation",
      "ffmpeg",
      "mqtt"
    ],
    "documentation": "https://github.com/greghesp/ha-bambulab",
    "iot_class": "local_push",
    "issue_tracker": "https://github.com/greghesp/ha-bambulab/issues",
    "ssdp": [
      {
        "st": "urn:bambulab-com:device:3dprinter:1"
      }
    ],
    "version": "2.0.15",
    "is_built_in": false
  },
  "data": {
    "config_entry": {
      "entry_id": "149d4dcc3453d1e3cb21625fb3349c4f",
      "version": 2,
      "minor_version": 1,
      "domain": "bambu_lab",
      "title": "**REDACTED**",
      "data": {
        "device_type": "P1S",
        "serial": "**REDACTED**"
      },
      "options": {
        "region": "Europe",
        "email": "**REDACTED**",
        "username": "**REDACTED**",
        "name": "Bambus",
        "host": "192.168.1.249",
        "local_mqtt": true,
        "auth_token": "**REDACTED**",
        "access_code": "**REDACTED**",
        "usage_hours": 21.0
      },
      "pref_disable_new_entities": false,
      "pref_disable_polling": false,
      "source": "user",
      "unique_id": null,
      "disabled_by": null
    },
    "push_all": {
      "upload": {
        "status": "idle",
        "progress": 0,
        "message": ""
      },
      "nozzle_temper": 24.25,
      "nozzle_target_temper": 0,
      "bed_temper": 21.1875,
      "bed_target_temper": 0,
      "chamber_temper": 5,
      "mc_print_stage": "1",
      "heatbreak_fan_speed": "0",
      "cooling_fan_speed": "0",
      "big_fan1_speed": "0",
      "big_fan2_speed": "0",
      "mc_percent": 100,
      "mc_remaining_time": 0,
      "ams_status": 0,
      "ams_rfid_status": 6,
      "hw_switch_state": 1,
      "spd_mag": 100,
      "spd_lvl": 2,
      "print_error": 0,
      "lifecycle": "product",
      "wifi_signal": "-64dBm",
      "gcode_state": "FINISH",
      "gcode_file_prepare_percent": "100",
      "queue_number": 0,
      "queue_total": 0,
      "queue_est": 0,
      "queue_sts": 0,
      "project_id": "61412289",
      "profile_id": "60272950",
      "task_id": "120698479",
      "subtask_id": "120698480",
      "subtask_name": "P1S Plug",
      "gcode_file": "P1S Plug.3mf",
      "stg": [],
      "stg_cur": 255,
      "print_type": "idle",
      "home_flag": 3294488,
      "mc_print_line_number": "122267",
      "mc_print_sub_stage": 0,
      "sdcard": true,
      "force_upgrade": false,
      "mess_production_state": "active",
      "layer_num": 30,
      "total_layer_num": 30,
      "s_obj": [],
      "filam_bak": [],
      "fan_gear": 0,
      "nozzle_diameter": "0.4",
      "nozzle_type": "stainless_steel",
      "upgrade_state": {
        "sequence_id": 0,
        "progress": "",
        "status": "",
        "consistency_request": false,
        "dis_state": 0,
        "err_code": 0,
        "force_upgrade": false,
        "message": "0%, 0B/s",
        "module": "",
        "new_version_state": 2,
        "cur_state_code": 0,
        "new_ver_list": []
      },
      "hms": [],
      "online": {
        "ahb": false,
        "rfid": false,
        "version": 881856849
      },
      "ams": {
        "ams": [],
        "ams_exist_bits": "0",
        "tray_exist_bits": "0",
        "tray_is_bbl_bits": "0",
        "tray_tar": "254",
        "tray_now": "254",
        "tray_pre": "254",
        "tray_read_done_bits": "0",
        "tray_reading_bits": "0",
        "version": 7,
        "insert_flag": true,
        "power_on_flag": false
      },
      "ipcam": {
        "ipcam_dev": "1",
        "ipcam_record": "enable",
        "timelapse": "disable",
        "resolution": "",
        "mode_bits": 3
      },
      "vt_tray": {
        "id": "254",
        "tag_uid": "0000000000000000",
        "tray_id_name": "",
        "tray_info_idx": "GFB00",
        "tray_type": "ABS",
        "tray_sub_brands": "",
        "tray_color": "BCBCBCFF",
        "tray_weight": "0",
        "tray_diameter": "0.00",
        "tray_temp": "0",
        "tray_time": "0",
        "bed_temp_type": "0",
        "bed_temp": "0",
        "nozzle_temp_max": "280",
        "nozzle_temp_min": "240",
        "xcam_info": "000000000000000000000000",
        "tray_uuid": "00000000000000000000000000000000",
        "remain": 0,
        "k": 0.014999999664723873,
        "n": 1
      },
      "lights_report": [
        {
          "node": "chamber_light",
          "mode": "on"
        }
      ],
      "command": "push_status",
      "msg": 0,
      "sequence_id": "1050"
    },
    "get_version": {
      "command": "get_version",
      "sequence_id": "0",
      "module": [
        {
          "name": "ota",
          "project_name": "C12",
          "sw_ver": "01.05.02.00",
          "hw_ver": "OTA",
          "sn": "**REDACTED**",
          "flag": 3
        },
        {
          "name": "esp32",
          "project_name": "C12",
          "sw_ver": "01.07.23.47",
          "hw_ver": "AP04",
          "sn": "**REDACTED**",
          "flag": 0
        },
        {
          "name": "mc",
          "project_name": "P1",
          "sw_ver": "00.00.22.39",
          "loader_ver": "00.00.00.28",
          "hw_ver": "MC07",
          "sn": "**REDACTED**",
          "flag": 0
        },
        {
          "name": "th",
          "project_name": "P1",
          "sw_ver": "00.00.07.12",
          "loader_ver": "00.00.00.14",
          "hw_ver": "TH09",
          "sn": "**REDACTED**",
          "flag": 0
        }
      ],
      "result": "success",
      "reason": ""
    }
  }
}

Log Extracts

2024-03-20 01:48:53.828 WARNING (MainThread) [homeassistant.components.light] None (<class 'custom_components.bambu_lab.light.BambuLabChamberLight'>) does not set supported color modes, this will stop working in Home Assistant Core 2025.3, please create a bug report at https://github.com/greghesp/ha-bambulab/issues
2024-03-20 02:18:24.040 WARNING (Thread-15) [homeassistant.helpers.entity] Updating state for light.bambus_chamber_light (<class 'custom_components.bambu_lab.light.BambuLabChamberLight'>) took 0.460 seconds. Please create a bug report at https://github.com/greghesp/ha-bambulab/issues
2024-03-20 02:18:42.331 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:42.331 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:42.331 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:43.334 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:43.334 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:43.334 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:43.337 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_31] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker
    work_item = work_queue.get(block=True)
2024-03-20 02:18:44.223 ERROR (Thread-14) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:44.224 ERROR (Thread-14) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:44.224 ERROR (Thread-14) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:44.434 ERROR (Thread-11 (mqtt_listen_thread)) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:44.434 ERROR (Thread-11 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:44.434 ERROR (Thread-11 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:45.337 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:45.341 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:45.342 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:46.343 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:46.344 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:46.344 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:48.346 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:48.347 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:48.347 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:49.349 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:49.349 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:49.349 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:51.351 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:51.352 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:51.352 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:52.353 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:52.353 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:52.353 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:54.356 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:54.356 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:54.356 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:55.358 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:55.359 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:55.359 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:57.361 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:57.361 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:57.361 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:18:58.362 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:18:58.363 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:18:58.363 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-20 02:19:00.366 ERROR (Thread-15) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-20 02:19:00.366 ERROR (Thread-15) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-20 02:19:00.366 ERROR (Thread-15) [custom_components.bambu_lab] Exception data: Event loop is closed

Other Information

No response

Takst3in commented 3 months ago

Crash log from v2.0.16-dev2 here;

2024-03-20 22:11:22.436 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task cancelling name='Task-6294' coro=<WebSocketCommonProtocol.close_connection() running at /usr/local/lib/python3.12/site-packages/websockets/legacy/protocol.py:1311> wait_for=<Future pending cb=[shield.._outer_done_callback() at /usr/local/lib/python3.12/asyncio/tasks.py:922, Task.task_wakeup()]> cb=[shield.._inner_done_callback() at /usr/local/lib/python3.12/asyncio/tasks.py:905]> 2024-03-20 22:11:22.444 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task cancelling name='Task-5580' coro=<BaseIoT.connect() running at /usr/local/lib/python3.12/site-packages/hass_nabucasa/iot_base.py:155> wait_for=<Task pending name='Task-187639' coro=<sleep() running at /usr/local/lib/python3.12/asyncio/tasks.py:665> wait_for= cb=[Task.task_wakeup()]>> 2024-03-20 22:11:22.444 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task cancelling name='Task-6305' coro=<TibberHome.rt_subscribe.._start() running at /usr/local/lib/python3.12/site-packages/tibber/home.py:439> wait_for=<Future pending cb=[shield.._outer_done_callback() at /usr/local/lib/python3.12/asyncio/tasks.py:922, Task.task_wakeup()]>> 2024-03-20 22:11:22.445 WARNING (MainThread) [homeassistant.runner] Task could not be canceled and was still running after shutdown: <Task cancelling name='Task-224' coro=<BaseIoT.connect() running at /usr/local/lib/python3.12/site-packages/hass_nabucasa/iot_base.py:136> wait_for=> 2024-03-20 22:11:22.446 ERROR (MainThread) [homeassistant] Error doing job: an error occurred during closing of asynchronous generator <async_generator object WebsocketsTransportBase.subscribe at 0x7f078dba8ca0> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/runner.py", line 188, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1961, in _run_once handle = heapq.heappop(self._scheduled) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

RuntimeError: aclose(): asynchronous generator is already running 2024-03-20 22:11:22.451 ERROR (MainThread) [homeassistant] Error doing job: an error occurred during closing of asynchronous generator <async_generator object AsyncClientSession.subscribe at 0x7f077a3ce640> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/runner.py", line 188, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1961, in _run_once handle = heapq.heappop(self._scheduled) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

RuntimeError: aclose(): asynchronous generator is already running 2024-03-20 22:11:22.455 ERROR (MainThread) [homeassistant] Error doing job: an error occurred during closing of asynchronous generator <async_generator object AsyncClientSession._subscribe at 0x7f078527cb80> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/runner.py", line 188, in run return loop.run_until_complete(setup_and_run_hass(runtime_config)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete self.run_forever() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever self._run_once() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1961, in _run_once handle = heapq.heappop(self._scheduled) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: list changed size during iteration

During handling of the above exception, another exception occurred:

RuntimeError: aclose(): asynchronous generator is already running 2024-03-20 22:11:22.495 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_15] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.518 CRITICAL (SyncWorker_15) [concurrent.futures] Exception in worker Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 82, in _worker work_item = work_queue.get_nowait() ^^^^^^^^^^^^^^^^^^^^^^^ _queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^ SystemExit 2024-03-20 22:11:22.534 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_60] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.552 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_24] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, *self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.593 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_61] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker work_item.run() File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/speedtestdotnet/coordinator.py", line 70, in update_data self.api.download() File "/usr/local/lib/python3.12/site-packages/speedtest.py", line 1569, in download prod_thread.join(timeout=0.001) File "/usr/local/lib/python3.12/threading.py", line 1151, in join self._wait_for_tstate_lock(timeout=max(timeout, 0)) File "/usr/local/lib/python3.12/threading.py", line 1167, in _wait_for_tstate_lock if lock.acquire(block, timeout): 2024-03-20 22:11:22.596 CRITICAL (SyncWorker_60) [concurrent.futures] Exception in worker Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 82, in _worker work_item = work_queue.get_nowait() ^^^^^^^^^^^^^^^^^^^^^^^ _queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^ SystemExit 2024-03-20 22:11:22.610 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_55] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.616 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_24] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, *self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.636 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_61] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.653 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_55] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.694 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_13] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, *self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker work_item.run() File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, self.kwargs) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 1814, in getMost results = self.performRequest(requestData) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 575, in performRequest res = self.request( File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 163, in request response = session.request(method, url, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrapsocket( File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 453, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_intls) File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 495, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File "/usr/local/lib/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( File "/usr/local/lib/python3.12/ssl.py", line 1042, in _create self.do_handshake() File "/usr/local/lib/python3.12/ssl.py", line 1320, in do_handshake self._sslobj.do_handshake() 2024-03-20 22:11:22.696 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_13] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, *self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker work_item.run() File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, self.kwargs) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 1814, in getMost results = self.performRequest(requestData) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 575, in performRequest res = self.request( File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 163, in request response = session.request(method, url, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrapsocket( File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 453, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_intls) File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 495, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File "/usr/local/lib/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( File "/usr/local/lib/python3.12/ssl.py", line 1042, in _create self.do_handshake() File "/usr/local/lib/python3.12/ssl.py", line 1320, in do_handshake self._sslobj.do_handshake() 2024-03-20 22:11:22.861 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_24] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, *self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:22.946 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_24] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:23.025 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_61] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:23.189 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_55] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, *self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:23.194 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_61] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:23.353 WARNING (Thread-138 (_do_shutdown)) [homeassistant.util.executor] Thread[SyncWorker_13] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker work_item.run() File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, self.kwargs) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 1814, in getMost results = self.performRequest(requestData) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 575, in performRequest res = self.request( File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 163, in request response = session.request(method, url, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrapsocket( File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 453, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_intls) File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 495, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File "/usr/local/lib/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( File "/usr/local/lib/python3.12/ssl.py", line 1042, in _create self.do_handshake() File "/usr/local/lib/python3.12/ssl.py", line 1320, in do_handshake self._sslobj.do_handshake() 2024-03-20 22:11:23.444 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_55] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) 2024-03-20 22:11:23.693 WARNING (MainThread) [homeassistant.util.executor] Thread[SyncWorker_13] is still running at shutdown: File "/usr/local/lib/python3.12/threading.py", line 1030, in _bootstrap self._bootstrap_inner() File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(*self._args, self._kwargs) File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 92, in _worker work_item.run() File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, self.kwargs) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 1814, in getMost results = self.performRequest(requestData) File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 575, in performRequest res = self.request( File "/usr/local/lib/python3.12/site-packages/pytapo/init.py", line 163, in request response = session.request(method, url, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/usr/local/lib/python3.12/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/usr/local/lib/python3.12/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/usr/local/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/usr/local/lib/python3.12/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrapsocket( File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 453, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_intls) File "/usr/local/lib/python3.12/site-packages/urllib3/util/ssl.py", line 495, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock) File "/usr/local/lib/python3.12/ssl.py", line 455, in wrap_socket return self.sslsocket_class._create( File "/usr/local/lib/python3.12/ssl.py", line 1042, in _create self.do_handshake() File "/usr/local/lib/python3.12/ssl.py", line 1320, in do_handshake self._sslobj.do_handshake() 2024-03-20 22:11:25.478 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:25.479 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:25.479 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:25.490 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:25.491 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:25.491 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:26.953 ERROR (Thread-7 (_thread_main)) [paho.mqtt.client] Caught exception in on_message: Event loop is closed 2024-03-20 22:11:26.955 ERROR (Thread-7 (_thread_main)) [root] Uncaught thread exception Traceback (most recent call last): File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner self.run() File "/usr/local/lib/python3.12/threading.py", line 1010, in run self._target(self._args, self._kwargs) File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 3591, in _thread_main self.loop_forever(retry_first_connection=True) File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 1756, in loop_forever rc = self._loop(timeout) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 1164, in _loop rc = self.loop_read() ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 1556, in loop_read rc = self._packet_read() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 2439, in _packet_read rc = self._packet_handle() ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 3033, in _packet_handle return self._handle_publish() ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 3327, in _handle_publish self._handle_on_message(message) File "/usr/local/lib/python3.12/site-packages/paho/mqtt/client.py", line 3570, in _handle_on_message on_message(self, self._userdata, message) File "/usr/src/homeassistant/homeassistant/components/mqtt/client.py", line 807, in _mqtt_on_message self.loop.call_soon_threadsafe(self._mqtt_handle_message, msg) File "/usr/local/lib/python3.12/asyncio/base_events.py", line 838, in call_soon_threadsafe self._check_closed() File "/usr/local/lib/python3.12/asyncio/base_events.py", line 540, in _check_closed raise RuntimeError('Event loop is closed') RuntimeError: Event loop is closed 2024-03-20 22:11:27.494 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:27.494 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:27.494 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:28.496 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:28.496 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:28.497 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:29.499 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:29.499 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:29.500 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:30.501 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:30.502 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:30.502 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:32.506 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:32.506 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:32.506 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:32.599 CRITICAL (SyncWorker_24) [concurrent.futures] Exception in worker Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 82, in _worker work_item = work_queue.get_nowait() ^^^^^^^^^^^^^^^^^^^^^^^ _queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^ SystemExit 2024-03-20 22:11:32.602 CRITICAL (SyncWorker_61) [concurrent.futures] Exception in worker Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 82, in _worker work_item = work_queue.get_nowait() ^^^^^^^^^^^^^^^^^^^^^^^ _queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^ SystemExit 2024-03-20 22:11:32.605 CRITICAL (SyncWorker_55) [concurrent.futures] Exception in worker Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 82, in _worker work_item = work_queue.get_nowait() ^^^^^^^^^^^^^^^^^^^^^^^ _queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 89, in _worker work_item = work_queue.get(block=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^ SystemExit 2024-03-20 22:11:33.135 ERROR (Thread-112 (mqtt_listen_thread)) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:33.135 ERROR (Thread-112 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:33.135 ERROR (Thread-112 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:33.508 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:33.509 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:33.509 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:34.510 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:34.510 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:34.511 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:35.157 ERROR (Thread-112 (mqtt_listen_thread)) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:35.158 ERROR (Thread-112 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:35.158 ERROR (Thread-112 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:36.515 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:36.515 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:36.516 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:37.517 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:37.517 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:37.517 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:38.518 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:38.519 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:38.519 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:40.521 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:40.522 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:40.522 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed 2024-03-20 22:11:41.523 ERROR (Thread-114) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data(): 2024-03-20 22:11:41.524 ERROR (Thread-114) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'> 2024-03-20 22:11:41.524 ERROR (Thread-114) [custom_components.bambu_lab] Exception data: Event loop is closed

bjeanes commented 3 months ago

I am also seeing this. Same pattern of exceptions of these three lines:

2024-03-21 21:54:14.975 ERROR (Thread-89223) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-21 21:54:14.976 ERROR (Thread-89223) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-21 21:54:14.976 ERROR (Thread-89223) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception data: Event loop is closed

However, I don't know if these exceptions are a symptom of a crash elsewhere or the cause.

johntdyer commented 3 months ago

Yea, I've had to disable tis for the time being because of the stability issues

bjeanes commented 3 months ago

I have too. I'm not sure why, but my HA is not restarting after crashing. Supervisor is not doing its duty it seems, and I have to SSH into the box and run ha core start regularly. So far, with Bambu disabled it hasn't crashed, but I only did it about an hour ago so I'm sitting tight to see. There is an exception I see in logs before Bambu coming from my Shelly integration that could be the root culprit, but if others are finding this with Bambu Labs integration, that is feeling more likely.

Takst3in commented 3 months ago

I have too. I'm not sure why, but my HA is not restarting after crashing. Supervisor is not doing its duty it seems, and I have to SSH into the box and run ha core start regularly. So far, with Bambu disabled it hasn't crashed, but I only did it about an hour ago so I'm sitting tight to see. There is an exception I see in logs before Bambu coming from my Shelly integration that could be the root culprit, but if others are finding this with Bambu Labs integration, that is feeling more likely.

This is correct, I also often needs to do an "core start" after the crash.

AdrianGarside commented 3 months ago

I am also seeing this. Same pattern of exceptions of these three lines:

2024-03-21 21:54:14.975 ERROR (Thread-89223) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-21 21:54:14.976 ERROR (Thread-89223) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-21 21:54:14.976 ERROR (Thread-89223) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception data: Event loop is closed

However, I don't know if these exceptions are a symptom of a crash elsewhere or the cause.

They are a symptom. Once the HA main thread gets into the bad state every time the bambu integration tells it we have new data we get an exception back.

AdrianGarside commented 3 months ago

For those hitting this, I need full debug logs to try and work out what the trigger is for this. I am not hitting it myself. I'll see if updating my HA version gets me the repro.

bjeanes commented 3 months ago

True as that may be, I haven't had a crash since disabling the integration.

AdrianGarside commented 3 months ago

I need full debug logs (not just debug for the integration) from someone that can readily reproduce it to see if i can work out what the initial trigger is.

Takst3in commented 3 months ago

I need full debug logs (not just debug for the integration) from someone that can readily reproduce it to see if i can work out what the initial trigger is.

I can reliable reproduce, but I'm at easter vacation right now, and I dont want to mess with the stability now when I'm not home (got it disabled atm). I can supply you these in 3-4 days if not anyone else beats me to it before then.

Thanks!

AdrianGarside commented 3 months ago

@Takst3in will you be able to get me some comprehensive (full HA) debug logs? So far with just debug logs for this integration I haven't been able to identify how the integration is causing this problem. And I have never hit this myself.

Takst3in commented 3 months ago

@Takst3in will you be able to get me some comprehensive (full HA) debug logs? So far with just debug logs for this integration I haven't been able to identify how the integration is causing this problem. And I have never hit this myself.

Managed to send you the logs direcly in a Message in the Home Assistant Community, since I were unsure about the sensitivity of these files.

pmetzke commented 3 months ago

I am also seeing this. Same pattern of exceptions of these three lines:

2024-03-21 21:54:14.975 ERROR (Thread-89223) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-21 21:54:14.976 ERROR (Thread-89223) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-21 21:54:14.976 ERROR (Thread-89223) [custom_components.bambu_lab] Exception data: Event loop is closed
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] An exception occurred calling async_set_updated_data():
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception type: <class 'RuntimeError'>
2024-03-21 21:54:15.205 ERROR (Thread-4 (mqtt_listen_thread)) [custom_components.bambu_lab] Exception data: Event loop is closed

However, I don't know if these exceptions are a symptom of a crash elsewhere or the cause.

I have the same experience with this add-on enabled. I have disabled it for now, but can enable it and try to grab more logs if needed.

I have a P1P (upgrade to P1S). I have been running this plug in since late Dec, and I didn't have any issues for quite a while.

AdrianGarside commented 3 months ago

Do you know what changed? Did you upgrade HA, upgrade the plugin? If either of those do you know what both the old and new versions were?

pmetzke commented 3 months ago

I don't have good details for you. It started back in Feb/early March, and I try to keep both HA and the addons up to date. I disabled several other addons to isolate, and this was the one that seemed to give me stability. I finally got some time to look into it, and realized other people were seeing something similar.

johntdyer commented 3 months ago

I don't have good details for you. It started back in Feb/early March, and I try to keep both HA and the addons up to date. I disabled several other addons to isolate, and this was the one that seemed to give me stability. I finally got some time to look into it, and realized other people were seeing something similar.

Exactly the same situation for me .... once I disabled this my stability returned to normal... as a test I re-enabled this and it crashed within an hour

boehser-enkel commented 2 months ago

Same for me but only when the printer prints. Do you need additional logs?

AdrianGarside commented 2 months ago

Please try this version: https://github.com/greghesp/ha-bambulab/releases/tag/v2.0.16-dev4

AdrianGarside commented 2 months ago

@Takst3in since you had the quickest / most consistent repro I'd appreciate if you could give this version a go. It fixes a threading issue that has a high chance of being the root cause of your issues.

Takst3in commented 2 months ago

@Takst3in since you had the quickest / most consistent repro I'd appreciate if you could give this version a go. It fixes a threading issue that has a high chance of being the root cause of your issues.

Installed 👍

I'll let it run overnight without debug first, then I'll try some debugging with ongoing prints in about 18hours time.

Takst3in commented 2 months ago

Still going strong, 7 hours in.. Also worth mentioning that my setup have been ported over to Proxmox this weekend, but if it stable, its stable.. I'll let it run till after work before stress testing(if active printing actually have something to say in this..)

AdrianGarside commented 2 months ago

I'm keeping my fingers crossed the threading issue was the majority of entirety of the recent instability. It's a little surprising that HA running in docker seemed to continue to be rock solid though.

Takst3in commented 2 months ago

It had 21h without any issue, before I had to take down the host for some bios modifications, after this, its stable for additional 9h.

It seems OK 👌 push it.. 🤞

Great & wonderful assistance on this annoying issue 💯

boehser-enkel commented 2 months ago

I'm keeping my fingers crossed the threading issue was the majority of entirety of the recent instability. It's a little surprising that HA running in docker seemed to continue to be rock solid though.

Not for my docker setup tho

AdrianGarside commented 2 months ago

Closing this as resolved. Please open a new issue (with debug logs) if you still hit any further instability to help me keep repros prior to this fix vs after.

AdrianGarside commented 2 months ago

Not for my docker setup tho I'd love to know why I never saw any instability. I hate bugs I never see.

Takst3in commented 2 months ago

@AdrianGarside is this fix included in 2.0.18?

(stable as f&$# now!) 😊

AdrianGarside commented 2 months ago

Yes

pmetzke commented 2 months ago

I’m stable now too. Thank you very much for fixing this.

pmetzke commented 2 months ago

Yes

Do you have a link to buy you a coffee or beer?

AdrianGarside commented 2 months ago

Yes :). https://ko-fi.com/adriangarside