search

alpine-docker / ollama

Minimal CPU-only Ollama Docker Image
https://hub.docker.com/repository/docker/alpine/ollama/general
MIT License
0 stars 0 forks source link

Include AVX Runner #1

Open kth8 opened 1 week ago

kth8 commented 1 week ago

without AVX runner the performance is going to be crippled as discussed in https://github.com/ollama/ollama/issues/2821

ozbillwang commented 1 week ago

yes, I am working on it.

The problem here is, I need use same Dockerfile, build for both ARM64 and AMD64.

https://github.com/alpine-docker/ollama/blob/master/Dockerfile#L10-L12

I commented out below lines

# In arm64 ollama/ollama image, there is no avx libraries and seems they are not must-have (#2903, #3891)
COPY --from=ollama /usr/lib/ollama/runners/cpu_avx /usr/lib/ollama/runners/cpu_avx
COPY --from=ollama /usr/lib/ollama/runners/cpu_avx2 /usr/lib/ollama/runners/cpu_avx2

But in ARM build, above two folders are not exist.

In Dockerfile , COPY doesn't support wildcard yet, I don't get folders, but files under destination folder

COPY --from=ollama /usr/lib/ollama/runners/cpu_* /usr/lib/ollama/runners/

Need find way to add them back