search

go-skynet / go-llama.cpp

LLama.cpp golang bindings
MIT License
626 stars 78 forks source link

add batched mode #253

Open mudler opened 9 months ago

mudler commented 9 months ago

https://github.com/ggerganov/llama.cpp/blob/master/examples/batched/batched.cpp