Add Cuda Graphs in CMakeList

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

4.34k stars 310 forks source link

Add Cuda Graphs in CMakeList #947

Closed Nexesenex closed 3 days ago

Nexesenex commented 5 days ago

Added by Slaren on LCPP after the recent refactor of LCPP.

https://github.com/ggerganov/llama.cpp/pull/8140

It must be for something (even if I saw the graphs in the KCPP logs already), so this PR allows to benefit from it now (or disable it!).

[x] I have read the contributing guidelines
Self-reported review complexity:
- [x] Low
- [ ] Medium
- [ ] High

LostRuins commented 3 days ago

This is actually incomplete and does not do anything. You need to add the compile definitions. I will merge it and fix it later.

Nexesenex commented 3 days ago

Oh, I didn't understand either why there was no definition. I missed to check into the other Cmakelists appeared with the refactor. Apologies². At least it came to your attention. :P