search

go-skynet / go-llama.cpp

LLama.cpp golang bindings
MIT License
626 stars 78 forks source link

Stop words don't seem to work #240

Closed 65a closed 9 months ago

65a commented 9 months ago

To reproduce, run the example from README.md with a reasonable model and ask "What is an animal similar to an alpaca?". The response should terminate generation including the word "llama" based on the configuration in the example llama.SetStopWords("llama"), but instead it will happily banter on about llama being a similar animal. This may be the code that is turning a Go []string into a char*, since it does something like &[]C.char, which is a pointer to a Go array of CStrings, which has a different memory layout than a char** of null terminated C strings.

I may be misunderstanding stop words here, or this isn't working as expected. This may also be a llama.cpp issue, it's not clear to me.

65a commented 9 months ago

Duplicate of #183. Trying some local change to see if it is actually a cgo issue.