search

SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
https://scisharp.github.io/LLamaSharp
MIT License
2.31k stars 304 forks source link

[Feature]: Support for Gemma2 #808

Closed tinkle-bell closed 2 days ago

tinkle-bell commented 2 weeks ago

Background & Description

I think the backend should be updated to support for gemma2.

API & Usage

No response

How to implement

Update the backend.

martindevans commented 2 weeks ago

There are still PRs open in llama.cpp for proper gemma support (e.g. https://github.com/ggerganov/llama.cpp/pull/8197) which I'm waiting on before starting a backend update :)

mhjabreel commented 2 weeks ago

The PR https://github.com/ggerganov/llama.cpp/pull/8197 is merged, do you have any ETA for this feature?

martindevans commented 2 weeks ago

I've started on a binary update, but it's held up at the moment due to an issue in the MacOS build pipeline.

SignalRT commented 2 weeks ago

810 is the PR where I'm working on it

martindevans commented 2 days ago

Gemma2 is supported now that 0.14.0 is released 🥳