[Request] llama.cpp-*-git

[jlo62]

Package:

https://aur.archlinux.org/pkgbase/llama.cpp-git

Purpose:

llama.cpp is a port of Facebook's llama model in C/C++, and supports running many large language models (llms).

This pkgbase builds the following accelerated llama.cpp packages:

    llama.cpp-git
    llama.cpp-cublas-git
    llama.cpp-clblas-git
    llama.cpp-hipblas-git
    llama.cpp-sycl-f16-git
    llama.cpp-sycl-f32-git
    llama.cpp-vulkan-git

Benefits:

These packages contain gpu accelerated builds, which can speed up runtime by a multiple

Building:

The git release is useful for following upstream, as the non-git versions are often outdated and (at least amds) often lag behind/are unusable

Copyright:

MIT

Expected Interest:

Many

Already available?

No

Unique request?

Yes

Banned package?

No

More information:

No response

[Technetium1]

Can be added when it compiles successfully: llama.cpp-git.log

[xiota]

Looks like a library that no other package currently uses.

[Technetium1]

@xiota runs as an interactive program as well as a server.

The AUR package may need to be updated due to https://github.com/ggerganov/llama.cpp/pull/7809