search

ollama / ollama

Get up and running with Llama 3.2, Mistral, Gemma 2, and other large language models.
https://ollama.com
MIT License
100.01k stars 7.97k forks source link

Missing ROCm Library Files In ollama-linux-amd64-rocm.tgz #7817

Open admpalma opened 5 days ago

admpalma commented 5 days ago

What is the issue?

Issue Description:

From Ollama v0.4.0 onwards, the libhipblas.so.2 and libhipblas.so.2.1.60102 files are no longer present in the ollama-linux-amd64-rocm.tgz archive. This makes the library unusable by the ollama instance.

Original Issue

Ollama v0.3.14:

libhipblas.so.2 -> libhipblas.so.2.1.60102
libhipblas.so.2.1.60102

Ollama v0.4.0:

libhipblas.so -> libhipblas.so.2 (broken link)

Found Evidence:

Looking at the build logs ("Run ./scripts/build_linux.sh" step):

v0.3.14:

libhipblas:

libhipblas.so.2 and libhipblas.so.2.1.60102 are copied to the linux-amd64-rocm/lib/ollama path:

linux-amd64-rocm/lib/ollama:
#43 2018.8 + cp -a /opt/rocm/lib/libhipblas.so.2 /opt/rocm/lib/libhipblas.so.2.1.60102 ../../dist/linux-amd64//../linux-amd64-rocm/lib/ollama
...
#43 2018.8 + cp /opt/rocm-6.1.2/lib/libhipblas.so.2.1.60102 ../../dist/linux-amd64//../linux-amd64-rocm/lib/ollama

v0.4.0:

libhipblas:

libhipblas.so.2 and libhipblas.so.2.1.60102 are now copied into linux-amd64/lib/ollama and only libhipblas.so is copied into linux-amd64-rocm/lib/ollama, causing the broken link.

linux-amd64/lib/ollama:
#19 1826.4 cp -af /opt/rocm/lib/libhipblas.so.2.1.60102 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
#19 1826.4 cp -af /opt/rocm/lib/libhipblas.so.2 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
linux-amd64-rocm/lib/ollama:
#19 1828.5 cp -af /opt/rocm/lib//libhipblas.so /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/

librocblas:

Additionally, the librocblas related files are copied into linux-amd64/lib/ollama when they should only be present in linux-amd64-rocm/lib/ollama:

linux-amd64/lib/ollama:
#19 1826.4 cp -af /opt/rocm/lib/librocblas.so.4.1.60102 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
#19 1827.7 cp -af /opt/rocm/lib/librocblas.so.4 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
linux-amd64-rocm/lib/ollama:
#19 1828.7 cp -af /opt/rocm/lib//librocblas.so /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/
#19 1828.7 cp -af /opt/rocm/lib//librocblas.so.4 /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/
#19 1828.8 cp -af /opt/rocm-6.1.2/lib//librocblas.so.4.1.60102 /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/

v0.4.4:

In v0.4.4 we can observe the same behavior as in v0.4.0:

libhipblas:

linux-amd64/lib/ollama:
#19 1292.4 cp -af /opt/rocm/lib/libhipblas.so.2.1.60102 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
#19 1292.4 cp -af /opt/rocm/lib/libhipblas.so.2 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
linux-amd64-rocm/lib/ollama:
#19 1425.8 cp -af /opt/rocm/lib//libhipblas.so /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/

librocblas:

linux-amd64/lib/ollama:
#19 1292.4 cp -af /opt/rocm/lib/librocblas.so.4.1.60102 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
...
#19 1293.7 cp -af /opt/rocm/lib/librocblas.so.4 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ollama/
linux-amd64-rocm/lib/ollama:
#19 1753.5 cp -af /opt/rocm/lib//librocblas.so /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/
#19 1753.6 cp -af /opt/rocm/lib//librocblas.so.4 /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/
#19 1753.6 cp -af /opt/rocm-6.1.2/lib//librocblas.so.4.1.60102 /go/src/github.com/ollama/ollama/dist/linux-amd64-rocm/lib/ollama/

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

v0.4.4

dhiltgen commented 4 days ago

The ollama-linux-amd64-rocm.tgz bundle is not self-sufficient and requires the payloads from ollama-linux-amd64.tgz to be complete. We'll get the dangling link fixed in the future so at least each tar file is self-consistent, but with the current packaging, you need both for AMD systems.

admpalma commented 4 days ago

you need both for AMD systems.

Indeed, to clarify, the Alpaca flatpak packages the contents of ollama-linux-amd64.tgz and ollama-linux-amd64-rocm.tgz onto separate folders, whereas the default Ollama installation method merges them.

Fixing the dangling link should therefore fix the original issue.