Closed andychenbruce closed 1 year ago
Hi andy, looks good but you need to upgrade the bindings.rs after you build the package. As per the bindgen userguide.
The resulting bindings will be written to $OUT_DIR/bindings.rs where $OUT_DIR is chosen by cargo and is something like ./target/debug/build/llm-chain-llama-sys-23ea4c1322828090/out/.
so copy the bindings.rs from there into to src/bindings.rs in llama-sys. Thanks!
👌
Looks like there's some build problems. Merge with main first before debugging, there were some problems in the main branch as well.
llama.cpp upstream had
tensor_split
converted from an array to a*const f32
which makes it a little more complicated since it doesn't have the traitSync
so any structures using it also aren'tSync
.https://github.com/ggerganov/llama.cpp/commit/ae178ab46bfd6ecb2422da5dad441a4e2fef8b7e
I think it is safe to explicitly mark them as so
as long as the data at the pointer is never mutated by llama.cpp but not sure.
Also had to remove the
Serialize, Deserialize
for the*const f32
from being derived but that didn't break anything so idk why they were there in the first place.