fix(modell_server): deprecated make arguments for llamacpp server

containers / ai-lab-recipes

Examples for building and running LLM services and applications locally with Podman

Apache License 2.0

103 stars 106 forks source link

Closed axel7083 closed 1 month ago

axel7083 commented 2 months ago

The LLAMA_CUBLAS argument is deprecated

lmilbaum commented 2 months ago

Would it make more sense to update the Renovate PR https://github.com/containers/ai-lab-recipes/pull/703 instead?

axel7083 commented 2 months ago

Would it make more sense to update the Renovate PR #703 instead?

Hi @lmilbaum there are two elements in this PR,

First the fix for the containerfile as mention in the description The LLAMA_CUBLAS argument is deprecated,
Second the bump of libraries

I did not see https://github.com/containers/ai-lab-recipes/pull/703 when I opened this PR this morning, thanks for informing me. I will be removing the bump of library to keep the focus on the fix.

Gregory-Pereira commented 2 months ago

Shouldn't this be:

CMAKE_ARGS="-DGGML_CUDA=on"

according to the docs. Guess im confused why we are turning it off when this variant is specifically cuda.