EricLBuehler / mistral.rs

Blazingly fast LLM inference.
MIT License
4.41k stars 308 forks source link

Pre-built binary for linux fails to launch with "error while loading shared libraries: libssl.so.1.1" #624

Open lxe opened 3 months ago

lxe commented 3 months ago

Describe the bug

Pre-built binary for linux fails to launch with

error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Reproduction steps:

$ curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.2/mistralrs-server-installer.sh | sh
downloading mistralrs-server 0.2.2 x86_64-unknown-linux-gnu
installing to /home/lxe/.cargo/bin
  mistralrs-server
  mistralrs-server-update
everything's installed!

To add $HOME/.cargo/bin to your PATH, either restart your shell or run:

    source $HOME/.cargo/env (sh, bash, zsh)
    source $HOME/.cargo/env.fish (fish)

$ source $HOME/.cargo/env

$ mistralrs-server
mistralrs-server: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

I already have libssl-dev:

$ sudo apt install libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libssl-dev is already the newest version (3.0.2-0ubuntu1.16).
0 upgraded, 0 newly installed, 0 to remove and 110 not upgraded.

System info:

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.4 LTS"
EricLBuehler commented 3 months ago

@ashleygwilliams do you know what may be causing this?

lxe commented 3 months ago

This is happening probably because I have OpenSSL3, but crate wants system to have OpenSSL1.1

lxe commented 3 months ago

Ubuntu 22.04 doesn't have OpenSSL1.1 in the repository, so I installed it manually:

$ wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.0g-2ubuntu4_amd64.deb
sudo dpkg -i libssl1.1_1.1.0g-2ubuntu4_amd64.deb

Now I get a different error:

$ mistralrs-server
mistralrs-server: /lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by mistralrs-server)
lxe commented 3 months ago

I'll just build it from source 🤷

EricLBuehler commented 3 months ago

Ok, thanks for the useful information. I looped in a maintainer of the library we use to create the prebuilt binaries, hopefully they can help!

EricLBuehler commented 3 months ago

@lxe was building from source successful?

lxe commented 3 months ago

@EricLBuehler yup no issues!

AshleySchaeffer commented 3 months ago

@EricLBuehler I see the project uses reqwest. If I were you, I'd just enable reqwest's rustls-tls feature:

https://docs.rs/reqwest/0.12.4/reqwest/index.html#optional-features

That way you don't need to rely on openssl at all. I'd consider enabling rustls-tls-webpki-roots too which uses this:

https://crates.io/crates/webpki-roots

EDIT: ahh it looks like hf-hub is going to be a problem.

EricLBuehler commented 3 months ago

Yeah, that's the reason. I considered forking hf-hub and modifying that to use rustls, but don't think it's worth it.

Repo-SK commented 3 months ago

Hey, I was able to get it working by installing the libssl like this:

wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb

If the link is expired then you can find the new one from: http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/?C=M;O=D

Just look for libssl1.1_1.1.1f-1ubuntu2.<LATEST_VERSION>_amd64.deb

To avoid this altogether I usually just bundle openssl into the binary:

[dependencies]
openssl = { version = "0.10.59", features = ["vendored"] }