search

esphome / issues

Issue Tracker for ESPHome
https://esphome.io/
290 stars 34 forks source link

Error when accessing logs on upgraded device: ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory #6189

Open poundy opened 3 weeks ago

poundy commented 3 weeks ago

The problem

I updated to 2024.8.0 today, was on 7.3 prior. I updated my online devices through the HA console after that. Now when I go to the LOGS on the device I get the following message:

INFO ESPHome 2024.8.0 INFO Reading configuration /config/esphome/displaytestnext.yaml... Traceback (most recent call last): File "/usr/local/bin/esphome", line 8, in <module> sys.exit(main()) ^^^^^^ File "/esphome/esphome/__main__.py", line 1014, in main return run_esphome(sys.argv) ^^^^^^^^^^^^^^^^^^^^^ File "/esphome/esphome/__main__.py", line 1001, in run_esphome rc = POST_CONFIG_ACTIONS[args.command](args, config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/esphome/esphome/__main__.py", line 481, in command_logs return show_logs(config, args, port) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/esphome/esphome/__main__.py", line 383, in show_logs from esphome.components.api.client import run_logs File "/esphome/esphome/components/api/client.py", line 8, in <module> from aioesphomeapi import APIClient File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/__init__.py", line 7, in <module> from .client import APIClient File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/client.py", line 91, in <module> from .connection import APIConnection, ConnectionParams, handle_timeout File "aioesphomeapi/connection.py", line 24, in init aioesphomeapi.connection File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/_frame_helper/__init__.py", line 3, in <module> from .noise import APINoiseFrameHelper File "aioesphomeapi/_frame_helper/noise.py", line 10, in init aioesphomeapi._frame_helper.noise File "/usr/local/lib/python3.11/dist-packages/chacha20poly1305_reuseable/__init__.py", line 7, in <module> from cryptography.hazmat.primitives.ciphers.aead import ( File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module> from cryptography.hazmat.primitives.ciphers.base import ( File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module> from cryptography.hazmat.bindings._rust import openssl as rust_openssl ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory

Which version of ESPHome has the issue?

2024.8.0

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

2024.7.4

What platform are you using?

ESP8266

Board

D1 Mini and others

Component causing the issue

LOGS

Example YAML snippet

All my devices now do this. ESP8266 and ESP32

Anything in the logs that might be useful for us?

INFO ESPHome 2024.8.0
INFO Reading configuration /config/esphome/displaytestnext.yaml...
Traceback (most recent call last):
  File "/usr/local/bin/esphome", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/esphome/esphome/__main__.py", line 1014, in main
    return run_esphome(sys.argv)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 1001, in run_esphome
    rc = POST_CONFIG_ACTIONS[args.command](args, config)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 481, in command_logs
    return show_logs(config, args, port)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 383, in show_logs
    from esphome.components.api.client import run_logs
  File "/esphome/esphome/components/api/client.py", line 8, in <module>
    from aioesphomeapi import APIClient
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/__init__.py", line 7, in <module>
    from .client import APIClient
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/client.py", line 91, in <module>
    from .connection import APIConnection, ConnectionParams, handle_timeout
  File "aioesphomeapi/connection.py", line 24, in init aioesphomeapi.connection
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/_frame_helper/__init__.py", line 3, in <module>
    from .noise import APINoiseFrameHelper
  File "aioesphomeapi/_frame_helper/noise.py", line 10, in init aioesphomeapi._frame_helper.noise
  File "/usr/local/lib/python3.11/dist-packages/chacha20poly1305_reuseable/__init__.py", line 7, in <module>
    from cryptography.hazmat.primitives.ciphers.aead import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module>
    from cryptography.hazmat.primitives.ciphers.base import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module>
    from cryptography.hazmat.bindings._rust import openssl as rust_openssl
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory

Additional information

No response

e-d-k commented 3 weeks ago

I have same issue on ESP32, Home Assistant 2024.8.2. Cant acces logs even on ESP8266 devices with ESPHome 2024.7.1, 2024.7.3.

Geo-Ron commented 3 weeks ago

Same here when reading the logs wireless:

INFO ESPHome 2024.8.0
INFO Reading configuration /config/esphome/deurbel-schakeling.yaml...
Traceback (most recent call last):
  File "/usr/local/bin/esphome", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/esphome/esphome/__main__.py", line 1014, in main
    return run_esphome(sys.argv)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 1001, in run_esphome
    rc = POST_CONFIG_ACTIONS[args.command](args, config)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 481, in command_logs
    return show_logs(config, args, port)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 383, in show_logs
    from esphome.components.api.client import run_logs
  File "/esphome/esphome/components/api/client.py", line 8, in <module>
    from aioesphomeapi import APIClient
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/__init__.py", line 7, in <module>
    from .client import APIClient
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/client.py", line 91, in <module>
    from .connection import APIConnection, ConnectionParams, handle_timeout
  File "aioesphomeapi/connection.py", line 24, in init aioesphomeapi.connection
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/_frame_helper/__init__.py", line 3, in <module>
    from .noise import APINoiseFrameHelper
  File "aioesphomeapi/_frame_helper/noise.py", line 10, in init aioesphomeapi._frame_helper.noise
  File "/usr/local/lib/python3.11/dist-packages/chacha20poly1305_reuseable/__init__.py", line 7, in <module>
    from cryptography.hazmat.primitives.ciphers.aead import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module>
    from cryptography.hazmat.primitives.ciphers.base import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module>
    from cryptography.hazmat.bindings._rust import openssl as rust_openssl
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory
randybb commented 3 weeks ago

No issues here, but I am running HA on x86-64. Have you tried to remove the esphome add-on and then install it back?

bartt-cz commented 3 weeks ago

Same here, no matter if its ESP8266, ESP32, bk7231 ...

INFO ESPHome 2024.8.0
INFO Reading configuration /config/esphome/cat-laser.yaml...
Traceback (most recent call last):
  File "/usr/local/bin/esphome", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/esphome/esphome/__main__.py", line 1014, in main
    return run_esphome(sys.argv)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 1001, in run_esphome
    rc = POST_CONFIG_ACTIONS[args.command](args, config)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 481, in command_logs
    return show_logs(config, args, port)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/esphome/esphome/__main__.py", line 383, in show_logs
    from esphome.components.api.client import run_logs
  File "/esphome/esphome/components/api/client.py", line 8, in <module>
    from aioesphomeapi import APIClient
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/__init__.py", line 7, in <module>
    from .client import APIClient
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/client.py", line 91, in <module>
    from .connection import APIConnection, ConnectionParams, handle_timeout
  File "aioesphomeapi/connection.py", line 24, in init aioesphomeapi.connection
  File "/usr/local/lib/python3.11/dist-packages/aioesphomeapi/_frame_helper/__init__.py", line 3, in <module>
    from .noise import APINoiseFrameHelper
  File "aioesphomeapi/_frame_helper/noise.py", line 10, in init aioesphomeapi._frame_helper.noise
  File "/usr/local/lib/python3.11/dist-packages/chacha20poly1305_reuseable/__init__.py", line 7, in <module>
    from cryptography.hazmat.primitives.ciphers.aead import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module>
    from cryptography.hazmat.primitives.ciphers.base import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module>
    from cryptography.hazmat.bindings._rust import openssl as rust_openssl
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory
rmayron commented 3 weeks ago

Hi folks

for what it's worth, I have the same issue with mix of devices (ESP32, ESP8266) after updating ESPHOME to 2024.8.0. I have updated most devices to 2024.8.0 before noticing the problem but still have one device on 2024.7.3 which exhibits the same issue as the other devices.(so the issue is not with the device firmware...)

Cheers - Ran

Szabka commented 3 weeks ago

Hi,

I use the docker edition and I also have this error. The docker image is based on Bookworm but there is no libssl1.1 library installed in it, maybe because it was dropped from the distro (too old maybe?) I successfully worked around the issue by installing the bullseye version inside the image. https://packages.debian.org/bullseye/armhf/libssl1.1/download

I hope that the maintainers will fix the image soon.

BR Szabolcs

orosam commented 3 weeks ago

Affects Home Assistant on Home Assistant Operating System as well (since it's also docker-based).

randybb commented 3 weeks ago

So all of you are running it on some arm platform? As I have mentioned, on x86-68 it is running fine (HA Addon and docker). Only thing change related to armv7 was this one https://github.com/esphome/esphome/pull/7022 which changed libssl-dev=3.0.11-1~deb12u2 -> libssl-dev=3.0.13-1~deb12u1 + https://github.com/esphome/esphome/pull/7021

It is strange, as it is pasing docker CI tests.

Maybe related https://www.reddit.com/r/debian/comments/1e11pal/package_is_available_on_a_mirror_but_not_on_the/

orosam commented 3 weeks ago

I'm running HAOS 13.1 on a RPi3, so armv7l here.

The esphome container has libssl3 (3.0.13), while cryptography seems to depend on libssl1.1 here:

~ $ docker container exec -it addon_5c53de3b_esphome python3 -c 'import cryptography.hazmat.primitives.ciphers.aead'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/__init__.py", line 11, in <module>
    from cryptography.hazmat.primitives.ciphers.base import (
  File "/usr/local/lib/python3.11/dist-packages/cryptography/hazmat/primitives/ciphers/base.py", line 10, in <module>
    from cryptography.hazmat.bindings._rust import openssl as rust_openssl
ImportError: libssl.so.1.1: cannot open shared object file: No such file or directory
bartt-cz commented 3 weeks ago

HAOS 13.1 on RPi4

electricalgorithm commented 3 weeks ago

Same here.

rmayron commented 3 weeks ago

as above, running on RPi 4 and having the issue

Core 2024.8.2 Supervisor 2024.08.0 Operating System 13.1 Frontend 20240809.0

Cheers, Ran

ngelotte commented 3 weeks ago

I am having the same issue. Using a Raspberry Pi with ESPHome 2024.8.0 Other details Core 2024.8.2 Supervisor 2024.08.0 Operating System 13.1 Frontend 20240809.0

Geo-Ron commented 3 weeks ago

No issues here, but I am running HA on x86-64. Have you tried to remove the esphome add-on and then install it back?

This has no effect. Same error remains.

6ffm70 commented 2 weeks ago

Also having the same issue on the devices I've updated, AND on the devices I've not updated.

Core 2024.8.3 Supervisor 2024.08.0 Operating System 13.1 Frontend 20240809.0 ESPHome 2024.8.0

Vartkat commented 2 weeks ago

same here on HA Supervised

michieleke commented 2 weeks ago

Same here on Raspberry Pi 400/Docker

Core 2024.8.2 Supervisor 2024.08.0 Frontend 20240809.0 ESPHome 2024.8.1

orosam commented 2 weeks ago

Yes, still present in 2024.8.1, the cryptography rust binding is still linked against openssl 1.1:

~ $ docker container exec -it addon_5c53de3b_esphome ldd /usr/local/lib/python3.11/dist-packages/cryptography/hazmat/bindings/_rust.abi3.so
        linux-vdso.so.1 (0x7ed2d000)
        libssl.so.1.1 => not found
        libcrypto.so.1.1 => not found
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x76f4f000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76c60000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76f4c000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76b49000)
        /lib/ld-linux-armhf.so.3 (0x76f58000)
JuliusSenkus commented 2 weeks ago

Same issue on ESPHome 2024.8.1, rpi 4 32-bit, OS 13.1, Core 2024.8.3, Supervisor 2024.08.0.

Maker39 commented 2 weeks ago

Same issue on Core 2024.8.3 Supervisor 2024.08.0 ESPHome 2024.8.1 (Aug 28 2024, 11:54:04)

poundy commented 2 weeks ago

confirmed that not addressed 2024.8.1 and is evident on legacy firmware devices or current firmware devices (so as Randybb intimated, it's the HA console not the ESPHome device). I now think ARMv7 is likely the commonality, I'm on an RPi4....

randybb commented 2 weeks ago

yes, this issue was not addressed yet and it is just due a missing lib in the armv7 docker container. other platforms are not affected.

greblosdier commented 2 weeks ago

I can confirm that the issue is still present with: Home Assistant (official armv7l image on Raspberry Pi 3) Core 2024.8.3 Supervisor 2024.08.0 Operating System 13.1 Frontend 20240809.0 ESPHome 2024.8.1

I ran docker container ls to show all running containers, found the ID for "esphome-hassio:2024.8.1" then ran docker exec -it [HEX ID NUMBER] bash to open a shell for the container. Using apt-get, I could see that libssl1 is unavailable in the repository. I manually downloaded and installed the package from the bullseye repository as indicated by [Szabka] using

curl http://ftp.us.debian.org/debian/pool/main/o/openssl/libssl1.1_1.1.1w-0+deb11u1_armhf.deb --output libssl1.1_1.1.1w-0+deb11u1_armhf.deb

and dpkg -i ./libssl1.1_1.1.1w-0+deb11u1_armhf.deb

This immediately fixed the ability to load the ESPHome device logs.

rmayron commented 2 weeks ago

thanks very much for that @greblosdier. The snag I have is not having dpkg installed. Is there a way of adding it manually? - or installing the missing library another way?

Many thanks Ran

mdionne0 commented 2 weeks ago

Same here.. on an Orange Pi5 with all 24 of my ESPhome nodes having the same problem... I need this fixed please.
image

technoo10201 commented 2 weeks ago

I can confirm that the issue is still present with: Home Assistant (official armv7l image on Raspberry Pi 3) Core 2024.8.3 Supervisor 2024.08.0 Operating System 13.1 Frontend 20240809.0 ESPHome 2024.8.1

I ran docker container ls to show all running containers, found the ID for "esphome-hassio:2024.8.1" then ran docker exec -it [HEX ID NUMBER] bash to open a shell for the container. Using apt-get, I could see that libssl1 is unavailable in the repository. I manually downloaded and installed the package from the bullseye repository as indicated by [Szabka] using

curl http://ftp.us.debian.org/debian/pool/main/o/openssl/libssl1.1_1.1.1w-0+deb11u1_armhf.deb --output libssl1.1_1.1.1w-0+deb11u1_armhf.deb

and dpkg -i ./libssl1.1_1.1.1w-0+deb11u1_armhf.deb

This immediately fixed the ability to load the ESPHome device logs.

I followed your recommendations, I confirm it works on my esphome (running in a docker container) ! Thanks for the tips (I will wait for a new docker image to deploy with the fix).

Have a nice day!

orosam commented 1 week ago

If I'm not mistaken, the underlying problem seems to be that the pre-built cryptography binary package obtained from https://www.piwheels.org/simple/cryptography/cryptography-43.0.0-cp37-abi3-linux_armv7l.whl is linked against openssl 1.1, while the base image has openssl 3.

Digging around a little, at the time of this post the cryptography 43.0.0 wheel seems to be disabled on piwheels, with skip_reason: "openssl issue" (see the Project JSON on the piwheels page). Interestingly, the wheel linked above can still be downloaded. There's an open pull request in piwheels that may or may not be related.

Meanwhile, I tried to rebuild the armv7 image (with forcing a cryptography rebuild from source), but got stuck building cryptography: error: package `cryptography-key-parsing v0.1.0 (...)` cannot be built because it requires rustc 1.65.0 or newer, while the currently active rustc version is 1.63.0 Which is correct, indeed: bookworm has rustc 1.63.

So, if I understand correctly, esphome needs cryptography == 43.0.0, but it cannot be built out-of-the-box on bookworm, on which the esphome docker image is based.

ssieb commented 1 week ago

I would recommend that you all re-install with a 64-bit system instead. 32-bit is really becoming unsupported in most places.

clowrey commented 1 week ago

I would recommend that you all re-install with a 64-bit system instead. 32-bit is really becoming unsupported in most places.

You are right, we just upgraded a 32 bit HAOS install on RPI 4 by making a full backup - flashing the SSD with the 64bit full HAOS image - then uploading the backup and now it works as expected. Just pick the -64 bit version for your RPI type https://github.com/home-assistant/operating-system/releases

poundy commented 1 week ago

just reporting in that 2023.8.3 hasn't addressed this.

Yeah I am finding the 32-bit install that I made a LONG time ago is apparently not getting me very far these days but I am resisting....

The-Kinky-Coder commented 1 week ago

Can confirm that the issue goes away when running on the 64 bit version. of Home Assistant.

I would recommend that you all re-install with a 64-bit system instead. 32-bit is really becoming unsupported in most places.

For me it appears I was running the 32 Bit version because I needed GPIO support and many, many moons ago the 32 Bit version was required for that, and it seems I never got round to swapping to 64 Bit in the years since.

orosam commented 1 week ago

I agree, on the long run probably moving to 64 bit would be the way.

On the other hand, the issue here is only that there is no pre-built cryptography wheel for armv7 linked against openssl3 (on piwheels).

I have just cobbled together some changes in the Dockerfile to build the cryptography wheel from source. It seems to work, logging and all, but I don't have a terribly complex setup.

jensbouma commented 1 week ago

On my RPI3, running HASS aarch64, ESPHome got somehow installed as armhf by default. Removing the armhf setting from the addon-config works for me, as it then installs the aarch64 image.

config.yaml

orosam commented 1 week ago

There is also a piwheels issue for cryptography 43.0.1 that is related. I don't know what would be the proper platform compatibility tag there (being cp37-abi3-linux_armv7l for both bullseye and bookworm).

orosam commented 1 week ago

... And, in the meantime as it also escaped my notice, folks at piwheels did compile a correct wheel, mentioned here: https://github.com/piwheels/packages/issues/464#issuecomment-2261391170

So now it should be a matter of downloading the wheel and installing it, no need to compile during docker build.

fmdvries commented 1 week ago

I have this same issue, RPI4, and I just updated to the last version of ESPHome and HA:

Core 2024.9.0 Supervisor 2024.08.0 Operating System 13.1 Frontend 20240904.0

Unfortunately I still get this error when requesting the logs for an ESP 8266 sensor that I'm trying to setup.

Switching to 64-bit may be a solution, but I don't really understand why that would be necessary. I'd rather not do that.

Would be great if someone could incorporate the workaround mentioned earlier in the 32 bit versions of HA & ESPHome.

Thanks!

orosam commented 1 week ago

Until then, one might try, on one's own responsibility, something like the oneliner below (for esphome 2024.8.3), executed from the Home Assistant terminal, to install the correct (==43.0.0 as of esphome 2024.8.3) cryptography from piwheels.

docker exec -it "$(docker ps --filter ancestor=ghcr.io/esphome/esphome-hassio:2024.8.3 --format '{{.ID}}')" bash -c "pip install --break-system-packages --force-reinstall --no-deps https://www.piwheels.org/cp311/cryptography-43.0.0-cp37-abi3-linux_armv7l.whl"

That said, I'm not responsible for any damage or injury caused to any person, property or anything else as a result of executing the command above :)

Use at your own risk, and beware that it will eventually break when executed on future esphome versions. (Hopefully - when this issue gets fixed. Or when the image starts to ship with a newer version. Or for many other reasons.)

Vartkat commented 6 days ago

@greblosdier solution did the trick. Hope the file is included in next release. (2024.8.3 here)

orosam commented 4 days ago

Use at your own risk, and beware that it will eventually break when executed on future esphome versions. (Hopefully - when this issue gets fixed. Or when the image starts to ship with a newer version. Or for many other reasons.)

Seems like the workaround is now included in PR #7426 🎉

gabrymed commented 15 hours ago

Same error for me. Waiting hopefully next update.

bartt-cz commented 12 hours ago

I also finally solved it by upgrading to the 64 bit version of HAOS. I was worried about it, but it was surprisingly easy - complete backup - install 64 bit version - restore backup. All done within half an hour, and everything works as before without any problems.