Open chenqiny opened 1 year ago
I opened an issue to llama.cpp. If it it built by cmake, then I will get same issue.
I used work around.
1.download llamacpp code
2. make
3. make libllama.so
4. overwrite libllama.so in llama-cpp-python
thanks @chenqiny. I also had the issue of illegal instruction. Your solution works!
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Expected Behavior
Load the model
Current Behavior
docker run --rm -it -p 9996:8000 -v /data/gguf/:/models -e MODEL=/models/llama-2-13b-chat.Q4_0.gguf ghcr.io/abetlen/llama-cpp-python:latest
python3 -m pip install -e . Obtaining file:///app Installing build dependencies ... done Checking if build backend supports build_editable ... done Getting requirements to build editable ... done Installing backend dependencies ... done Preparing editable metadata (pyproject.toml) ... done Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.11/site-packages (from llama_cpp_python==0.2.7) (4.8.0) Requirement already satisfied: numpy>=1.20.0 in /usr/local/lib/python3.11/site-packages (from llama_cpp_python==0.2.7) (1.26.0) Requirement already satisfied: diskcache>=5.6.1 in /usr/local/lib/python3.11/site-packages (from llama_cpp_python==0.2.7) (5.6.3) Building wheels for collected packages: llama_cpp_python Building editable for llama_cpp_python (pyproject.toml) ... done Created wheel for llama_cpp_python: filename=llama_cpp_python-0.2.7-cp311-cp311-manylinux_2_31_x86_64.whl size=911317 sha256=b77877c90bdba00e257432c49978a075519f5818f17e14ecc00db21c1fd6998c Stored in directory: /tmp/pip-ephem-wheel-cache-ivqpfggy/wheels/57/0f/98/bb57b2b57b95807699b822a35c022f139d38a02c27922f27ce Successfully built llama_cpp_python Installing collected packages: llama_cpp_python Attempting uninstall: llama_cpp_python Found existing installation: llama_cpp_python 0.2.7 Uninstalling llama_cpp_python-0.2.7: Successfully uninstalled llama_cpp_python-0.2.7 Successfully installed llama_cpp_python-0.2.7 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Illegal instruction (core dumped)
Environment and Context
Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.
$ lscpu
$ uname -a
Python 3.11.5 (main, Sep 20 2023, 11:03:59) [GCC 10.2.1 20210110] on linux
Failure Information (for bugs)
Illegal instruction (core dumped)
Steps to Reproduce
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
Note: Many issues seem to be regarding functional or performance issues / differences with
llama.cpp
. In these cases we need to confirm that you're comparing against the version ofllama.cpp
that was built with your python package, and which parameters you're passing to the context.Try the following:
git clone https://github.com/abetlen/llama-cpp-python
cd llama-cpp-python
rm -rf _skbuild/
# delete any old buildspython setup.py develop
cd ./vendor/llama.cpp
cmake
llama.cpp./main
with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, log an issue with llama.cppI tried it, then I got
root@51b054c89440:/work/llama-cpp-python/vendor/llama.cpp/build/bin# ./main Log start main: warning: changing RoPE frequency base to 0 (default 10000.0) main: warning: scaling RoPE frequency by 0 (default 1.0) main: build = 1271 (a98b163) main: built with cc (Debian 12.2.0-14) 12.2.0 for x86_64-linux-gnu main: seed = 1695717956 Illegal instruction (core dumped)
Failure Logs
Please include any relevant log snippets or files. If it works under one configuration but not under another, please provide logs for both configurations and their corresponding outputs so it is easy to see where behavior changes.
Also, please try to avoid using screenshots if at all possible. Instead, copy/paste the console output and use Github's markdown to cleanly format your logs for easy readability.