fix: safer loop bounds for TokenizeString, update golang to 1.21 - Githubissues

go-skynet / go-llama.cpp

LLama.cpp golang bindings

MIT License

650 stars 79 forks source link

fix: safer loop bounds for TokenizeString, update golang to 1.21 #177

Closed dave-gray101 closed 1 year ago

dave-gray101 commented 1 year ago

Still do not know how to reproduce Luna's crash, but the most likely cause is this loop.

Bump golang version to 1.21 for min() builtin
Use the smaller of the returned length and the actual length of the returned token slice
Since the returned length might be negative in error cases, this gives us a free way to skip the for loop as well, since -N < 0.