Bug: llama-server cannot open shared object file libllama.so after building from SYCL-Dockerfile

[HumerousGorgon]

What happened?

After building the llama-server-intel.Dockerfile into an image and running with the correct docker run args, every single attempt ends in : /llama-server: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

Building the CLI version results in a prompt being able to be sent and a response generated. I have tried using --no-cache on the docker build operation, in case previously cached file were messing with things.

I used a bit of a hodge-podge command, working off of the standard options needed in the llama-server dockerfile (non SYCL).

Name and Version

Version: llama-server-intel.Dockerfile. Command run: docker run -it -p 11434:11434 -v /mnt/user/models/model-files/:/app/models/ --device /dev /dri/renderD128:/dev/dri/renderD128 --device /dev/dri/card0:/dev/dri/card0 llama-server-cpp-sycl -m "/app/models/Meta-Llama-3-8B-Instruct-fp16.ggu f" -n 2048 -e -ngl 33 --host 0.0.0.0 --port 11434

What operating system are you seeing the problem on?

Linux

Relevant log output

/llama-server: error while loading shared libraries: libllama.so: cannot open shared object file: No such file or directory

[jboero]

A merge switched the Docker builds from make to cmake and also switched from static builds to shared libs. I too just noticed this and it affected some automation. You may want to switch your docker file.

https://github.com/ggerganov/llama.cpp/pull/9213

[HumerousGorgon]

A merge switched the Docker builds from make to cmake and also switched from static builds to shared libs. I too just noticed this and it affected some automation. You may want to switch your docker file.

9213

You mean switch it with a docker file from a commit prior to these changes?

[jboero]

You mean switch it with a docker file from a commit prior to these changes?

Either that or rewrite your own using make. In fact it might be handy to keep both versions or a variable for making static or shared lib versions? Similar to how CUDA version is set, these dockerfiles require a bit of customization.

[HumerousGorgon]

You mean switch it with a docker file from a commit prior to these changes?

Either that or rewrite your own using make. In fact it might be handy to keep both versions or a variable for making static or shared lib versions? Similar to how CUDA version is set, these dockerfiles require a bit of customization.

I'll have to work out how to make it work with make instead of cmake. I did try to change out the cmake and make terms in the docker file, but it freaked out, I'm assuming from the launch options. I'll look into it more later on.

Any idea where I might be able to start learning about where to go with this? I appreciate your help and advice.

[jboero]

Tags aren't easy to follow but here is a copy from 2 months ago you can try instead. You should just be able to keep a copy of this locally (outside the repo even) and use it like normal against the latest code. If you're trying to build your own custom Dockerfile it may be easier to interactively launch the build container with a shell and build manually until you can the build process the way you like. Note I use Podman just as well as Docker here, and Docker is deprecating certain build options..

https://github.com/ggerganov/llama.cpp/blob/b3598/.devops/llama-server-cuda.Dockerfile

[slaren]

9213 didn't change the SYCL images, only the CUDA images. The SYCL backend cannot be built with make, it requires cmake. This was probably broken when the build system was revamped.

Possible fixes could be to copy the dynamic libraries to the runtime image like the CUDA image does, or add -DBUILD_SHARED_LIBS=OFF to the cmake configure command line to disable dynamic linking.

[HumerousGorgon]

9213 didn't change the SYCL images, only the CUDA images. The SYCL backend cannot be built with make, it requires cmake. This was probably broken when the build system was revamped.

Possible fixes could be to copy the dynamic libraries to the runtime image like the CUDA image does, or add -DBUILD_SHARED_LIBS=OFF to the cmake configure command line to disable dynamic linking.

This definitely fixed the building issue for me, thank you very much. Another question for you: I seem to be having issues loading models that end up utilising more than 4GB of VRAM. I did read that the Arc series GPUs had issues with >4GB VRAM allocations but thought that this had been fixed. Do you have any advice on this issue? Thank you for your help!

[jboero]

I don't have an ARC to test anymore but I remember resizable BAR (reBAR) was basically a requirement for ARC GPUs. Your motherboard must support it and it must be enabled. Check that?

On Fri, Sep 13, 2024, 04:29 HumerousGorgon @.***> wrote:

9213 https://github.com/ggerganov/llama.cpp/pull/9213 didn't change

the SYCL images, only the CUDA images. The SYCL backend cannot be built with make, it requires cmake. This was probably broken when the build system was revamped.

Possible fixes could be to copy the dynamic libraries to the runtime image like the CUDA image does, or add -DBUILD_SHARED_LIBS=OFF to the cmake configure command line to disable dynamic linking.

This definitely fixed the building issue for me, thank you very much. Another question for you: I seem to be having issues loading models that end up utilising more than 4GB of VRAM. I did read that the Arc series GPUs had issues with >4GB VRAM allocations but thought that this had been fixed. Do you have any advice on this issue? Thank you for your help!

— Reply to this email directly, view it on GitHub https://github.com/ggerganov/llama.cpp/issues/9416#issuecomment-2347965632, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABZP3DGZMRHEDY7XRHI2LM3ZWJLY3AVCNFSM6AAAAABN7HEYD2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBXHE3DKNRTGI . You are receiving this because you commented.Message ID: <ggerganov/llama. @.***>

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.