Open siilats opened 11 months ago
use
pip uninstall llama-cpp-python -y CMAKE_ARGS="-DLLAMA_METAL=on" pip install -U llama-cpp-python --no-cache-dir pip install 'llama-cpp-python[server]'
to install metal version. only works on 4_0 quant models
use
to install metal version. only works on 4_0 quant models