Seeing Issue why trying to load

su77ungr / CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware

Apache License 2.0

230 stars 31 forks source link

Seeing Issue why trying to load #75

Closed keshavjha1 closed 1 year ago

keshavjha1 commented 1 year ago

I am facing below problem while running startLLM on Linux/Mac machine

(.venv) ke2@t2:~/projects/CASALIOY$ python3 casalioy/startLLM.py found local model at models/sentence-transformers/all-MiniLM-L6-v2 found local model at models/eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin llama.cpp: loading model from models/eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin terminate called after throwing an instance of 'std::runtime_error' what(): read error: Is a directory Aborted (core dumped)

su77ungr commented 1 year ago

What are your machines' specs?

Also this might be an error caused by wrongly specified paths. It tries to run a directory.

Might be fixed with new PRs. #72. Testing it now.

hippalectryon-0 commented 1 year ago

Same issue as #71 , should be fixed in #72 . Pull master and retry

su77ungr commented 1 year ago

Has this been resolved?