search

utilityai / llama-cpp-rs

155 stars 47 forks source link

`LlamaModel::get_chat_template` panics if the buffer size passed is too small for the template to fit in #411

Closed babichjacob closed 3 months ago

babichjacob commented 3 months ago

Reproduction in https://github.com/babichjacob/llama-cpp-rs/commit/35c1de9970e35256dee592391da879a99ad34113 specifically https://github.com/babichjacob/llama-cpp-rs/commit/35c1de9970e35256dee592391da879a99ad34113#diff-d7652fc11ef443b44fefcec061161ffc10fa5b19332c1d345cd3ad47465786e8R26-R35

So, the code shows loading LlamaModel (any model / GGUF will do) then trying to use theget_chat_template method with an insufficient buffer size (just 15 bytes long). This results in a panic

thread 'main' panicked at llama-cpp-2\src\model.rs:384:13:
assertion `left == right` failed: llama.cpp guarantees that the returned int 344 is the length of the string 14 but that was not the case
  left: 344
 right: 14

but it could be changed to be an Err so calling code can recover: there isn't a memory vulnerability since llama.cpp won't write outside of the buffer.