search

clp-research / clembench

A Framework for the Systematic Evaluation of Chat-Optimized Language Models as Conversational Agents and an Extensible Benchmark
MIT License
19 stars 26 forks source link

New model entries for May 2024 #92

Closed Gnurro closed 1 month ago

Gnurro commented 1 month ago

Adds HF model entries: Swallow-70b-instruct-v0.1, Phi-3-mini-128k-instruct, gemma-1.1-2b-it, gemma-1.1-7b-it, Starling-LM-7B-beta, codegemma-7b-it and recurrentgemma-2b-it to the model registry. Also adds new quantized GGUF model registry entries: Meta-Llama-3-70B-Instruct-GGUF-q4, Meta-Llama-3-70B-Instruct-GGUF-q8, c4ai-command-r-plus-GGUF-q4 and c4ai-command-r-plus-GGUF-q8. (Might be more to come.) Also includes a code addition to the llama.cpp backend to load sharded GGUF files from HuggingFace - this code change relies on an edit to the llama-cpp-python library, for which I've opened a PR on the corresponding repository: https://github.com/abetlen/llama-cpp-python/pull/1457 Adds messages list cleaning to llama.cpp backend generation calls.

Gnurro commented 1 month ago

We can merge this now, to get the llama.cpp backend code additions onto the main repo, I'll open a new PR for more potential model additions then. Do note that these will not work without my patch to llama-cpp-python, which may take a while to get merged and added to a PyPI release of llama-cpp-python. See PR linked in the first comment.