Control Vectors - Githubissues

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

MIT License

3.19k stars 234 forks source link

Control Vectors #442

Open acidbubbles opened 1 month ago

acidbubbles commented 1 month ago

There is hype right now for control vectors. It looks like an interesting way to "steer" the LLM without relying on prompting, saving tokens and increasing instructions strength.

Read more on this on these implementations:

llama.cpp: https://github.com/ggerganov/llama.cpp/pull/5970
oobabooga: https://github.com/Hellisotherpeople/llm_steer-oobabooga?tab=readme-ov-file