simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp
Apache License 2.0
139 stars 20 forks source link

Document different installation options #3

Closed simonw closed 1 year ago

simonw commented 1 year ago

https://github.com/simonw/llm-llama-cpp#installation

https://github.com/simonw/llm-llama-cpp/blob/1ac579fde696f26a19f21a9b315a3ffb3b3b4f16/pyproject.toml#L25-L29

Maybe add a llm-llama-cpp[compile] optional one that uses llama-cpp-python directly but only works if you have a C compiler.

simonw commented 1 year ago

My wheel probably only works if you have Python 3.11 - which you should do if you installed llm using Homebrew: https://github.com/Homebrew/homebrew-core/blob/ad30bf55d4cd894e71d3c1618ba150b81b06ce33/Formula/llm.rb#L24

simonw commented 1 year ago

This is made harder since llm install -e '.[mac_arm64]' doesn't work in stable at the moment:

Also, the wheel I built actually only works on Python 3.11 - so it's even more specific.

simonw commented 1 year ago

I could ship a special package just so this works:

llm install llm-llama-cpp-macos-arm-python311
simonw commented 1 year ago

Might need to ship:

I don't yet have a great pattern for shipping multiple releases from a single repo though.

simonw commented 1 year ago

Or, easier: ship llm-llama-cpp-base and llm-llama-cpp and tell Mac M1/M2 users with Homebrew's Python and no C compiler to run:

llm install llm-llama-cpp-base https://static.simonwillison.net/static/2023/llama_cpp_python-0.1.77-cp311-cp311-macosx_13_0_arm64.whl

But that doesn't make sense, because it's not possible to install Homebrew without installing Xcode first. So I'm definitely over-thinking this.

simonw commented 1 year ago

Absolute simplest option: leave llama-cpp-python out of the dependencies and in the installation instructions say:

llm install llm-llama-cpp llama-cpp-python 

Or if you're on an M1/M2 Mac using LLM installed from Homebrew:

llm install llm-llama-cpp https://static.simonwillison.net/static/2023/llama_cpp_python-0.1.77-cp311-cp311-macosx_13_0_arm64.whl