Closed siddhsql closed 1 year ago
if it helps, I stepped through the code in a debugger and it runs into a problem here:
def llama_init_from_file(
path_model: bytes, params: llama_context_params
) -> llama_context_p:
return _lib.llama_init_from_file(path_model, params)
upon returning from this function, the following assertion fails in llama.py
:
assert self.ctx is not None
the assertion fails under debugger but when running from command line the assertions are disabled and so the program continues and crashes later on.
adding some more notes for self: we do see this line in the output of llama-cpp-python
: https://github.com/ggerganov/llama.cpp/blob/5c64a0952ee58b2d742ee84e8e3d43cce5d366db/llama.cpp#L2501
fprintf(stderr, "%s: kv self size = %7.2f MB\n", __func__, memory_size / 1024.0 / 1024.0);
so then the only code left is what comes afterwards... and it does work when running llama.cpp
directly
it beats me. I added this code just before returning ctx
from llama.cpp
:
fprintf(stderr, "debug: returning ctx\n");
return ctx;
and I see this in the output console:
llama_init_from_file: kv self size = 256.00 MB
debug: returning ctx
and yet self.ctx
is None
in:
assert self.ctx is not None
how is this even possible?
running docker image errors out as well:
% docker run --rm -it -p 8000:8000 -v $PWD/models:/models -e MODEL=/models/$MODEL_NAME ghcr.io/abetlen/llama-cpp-python:latest
llama.cpp: loading model from /models/vicuna-7b-1.1.ggmlv3.q5_1.bin
Illegal instruction
am i the only one with this problem? can't be.
Illegal instruction usually indicates that a binary code is compiled for the wrong architecture, i.e. this is a compiler configuration issue.
correct. it seems the docker image has precompiled binary of llama.cpp for different architecture. also see: https://github.com/ggerganov/llama.cpp/issues/537 anyway the problem persists. i was trying to use docker as an alternative to see if that works. can someone please help me here? i can't believe i am the only one having this problem.
On Thu, Jun 8, 2023 at 8:17 AM Gary Mulder @.***> wrote:
Illegal instruction usually indicates that a binary code is compiled for the wrong architecture, i.e. this is a compiler configuration issue.
— Reply to this email directly, view it on GitHub https://github.com/abetlen/llama-cpp-python/issues/342#issuecomment-1582793164, or unsubscribe https://github.com/notifications/unsubscribe-auth/A6NWEK7NMBTCN6ADYPT6ZZ3XKHUG5ANCNFSM6AAAAAAY6QAGOQ . You are receiving this because you authored the thread.Message ID: @.***>
adding some more info to help anyone who runs into this problem: i tried pyllamacpp and it works (nevermind the garbage output):
% pyllamacpp ~/llm/llama-cpp-python/models/vicuna-7b-1.1.ggmlv3.q5_1.bin
██████╗ ██╗ ██╗██╗ ██╗ █████╗ ███╗ ███╗ █████╗ ██████╗██████╗ ██████╗
██╔══██╗╚██╗ ██╔╝██║ ██║ ██╔══██╗████╗ ████║██╔══██╗██╔════╝██╔══██╗██╔══██╗
██████╔╝ ╚████╔╝ ██║ ██║ ███████║██╔████╔██║███████║██║ ██████╔╝██████╔╝
██╔═══╝ ╚██╔╝ ██║ ██║ ██╔══██║██║╚██╔╝██║██╔══██║██║ ██╔═══╝ ██╔═══╝
██║ ██║ ███████╗███████╗██║ ██║██║ ╚═╝ ██║██║ ██║╚██████╗██║ ██║
╚═╝ ╚═╝ ╚══════╝╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝
PyLLaMACpp
A simple Command Line Interface to test the package
Version: 2.4.1
=========================================================================================
[+] Running model `/Users/xxx/llm/llama-cpp-python/models/vicuna-7b-1.1.ggmlv3.q5_1.bin`
[+] LLaMA context params: `{}`
[+] GPT params: `{}`
llama.cpp: loading model from /Users/xxx/llm/llama-cpp-python/models/vicuna-7b-1.1.ggmlv3.q5_1.bin
llama_model_load_internal: format = ggjt v3 (latest)
llama_model_load_internal: n_vocab = 32000
llama_model_load_internal: n_ctx = 512
llama_model_load_internal: n_embd = 4096
llama_model_load_internal: n_mult = 256
llama_model_load_internal: n_head = 32
llama_model_load_internal: n_layer = 32
llama_model_load_internal: n_rot = 128
llama_model_load_internal: ftype = 9 (mostly Q5_1)
llama_model_load_internal: n_ff = 11008
llama_model_load_internal: n_parts = 1
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 0.07 MB
llama_model_load_internal: mem required = 6612.59 MB (+ 2052.00 MB per state)
.
llama_init_from_file: kv self size = 512.00 MB
...
[+] Press Ctrl+C to Stop ...
...
You: who is the president of usa?
AI: class MQL5_CALLBACK_TYPE
{
public:
virtual bool OnTick() = 0;
};
class MQL5_EVENT_TYPE
{
public:
virtual void Process() = 0;
};
```scss
class MQL5_ON_TRADING_EVENT_TYPE : public MQL5_EVENT_TYPE
{
public:
void Process();
};
class MQL5_ON_ORDER_EVENT_TYPE : public MQL5_EVENT_TYPE
{
public:
void Process();
};
class MQL5_ON_TAKE_PROFIT_EVENT_TYPE : public MQL5_EVENT_TYPE
{
public:
void Process();
};
class MQL5_ON_STOP_LOSS_EVENT_TYPE : public MQL5_EVENT_TYPE
{
public:
void Process();
};
class MQL5_ON_HEDGE_EVENT_TYPE : public MQL5_EVENT_TYPE
{
public:
void Process();
};
class MQL5_ON_TRADE_END_EVENT_TYPE : public MQL5_EVENT_TYPE
{
public:
void Process();
};
//...
// Register events MQL5_ON_TRADING_EVENT_REGISTER(class MQL5_ON_TRADING_EVENT_TYPE); MQL5_ON_ORDER_EVENT_REGISTER(class MQL5_ON_ORDER_EVENT_TYPE); MQL5_ON_TAKE_PROFIT_EVENT_REGISTER(class MQL5_ON_TAKE_PROFIT_EVENT_TYPE); MQL5_ON_STOP_LOSS_EVENT_REGISTER(class MQL5_ON_STOP_LOSS_EVENT_TYPE); MQL5_ON_HEDGE_EVENT_REGISTER(class MQL5_ON_HEDGE_EVENT_TYPE);QL5_ON_TRADE_END_EVENT_REGISTER(class MQL5_ON_TRADE_END_EVENT_TYPE);//... You:
Closing please reopen if the problem is reproducible with the latest llama-cpp-python
which includes an updated llama.cpp
Prerequisites
Please answer the following questions for yourself before submitting an issue.
Observed Behavior
I am running v0.1.57 of the program with model weights from https://huggingface.co/TheBloke/vicuna-7B-1.1-GGML on an intel based MacOS with 16GB RAM and 6GB of used memory. I have been able to install the application but when I try to run it I get this:
The Python interpreter just crashes with a
SIGABRT
. there is no traceback printed on the screen.Expected Behavior
no error
Environment and Context
Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.
I am running on Mac OS Ventura with intel based 6 core CPU and 16GB of RAM. 6GB is used.
$ uname -a
Failure Information (for bugs)
https://gist.github.com/siddhsql/ea1d8b0289896a7a0748504f6802c8ac
Steps to Reproduce
see above.
Try the following:
git clone https://github.com/abetlen/llama-cpp-python
cd llama-cpp-python
rm -rf _skbuild/
# delete any old buildspython setup.py develop
cd ./vendor/llama.cpp
cmake
llama.cpp./main
with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, log an issue with llama.cppI did try this and it works. please see below:
Failure Logs
https://gist.github.com/siddhsql/ea1d8b0289896a7a0748504f6802c8ac