iboB
commented
3 months ago - I noted the revision of llama.cpp
- and didn't note the one from ggml. We aren't using raw ggml functions yet, so the changes there should not be relevant (again, yet!)
- I updated our forks and pulled the changes into the repo
- The configure failed, which led to https://github.com/ggerganov/llama.cpp/pull/8787
- Now the repo points to the branch for this PR
- Successful configure, build, and tests
- Though there are new warnings: note to make a PR about them. It is not critical though
- I opened the commit log of llama.cpp and started going through all commits from our previous version to the current, looking for something relevant and taking notes:
- a lot of commits were disregarded based on commit message
- starts with "ggml: " - not relevant (would be handled from the ggml submodule)
- specific backend - not relevant (yet?)
- python integration - not relevant
- etc... Using judgement here :)
- https://github.com/alpaca-core/llama.cpp/commit/938943cdbf4dd79005a394d732bc226f9e34e0ff - optimize headers
- https://github.com/alpaca-core/llama.cpp/commit/96952e7181929c6001b2bc69a33f240de731cc3a - may be relevant for our chat format reimplementation
- a lot of commits were disregarded based on commit message
- Just in case rechecked the history of:
llama.cpp/commonreimplemented by usllama.cpp/mainlook for new features- https://github.com/alpaca-core/llama.cpp/commit/6af51c0d96e6268769fc05c98d5b1a5e832c0017 - make sure we don't run without tokens
Also add relevant changes to our inference code if needed
Example issue for #33