su77ungr / CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware

Apache License 2.0

229 stars 32 forks source link

I was able to ask 3 questions and got a `GGML_ASSERT:` #28

Closed alxspiker closed 1 year ago

alxspiker commented 1 year ago

          I was able to ask 3 questions and got a `GGML_ASSERT: C:\Users\Haley The Retard\AppData\Local\Temp\pip-install-m9k6bx9s\llama-cpp-python_e02ecdc8e7e1464e99540ce48153ff94\vendor\llama.cpp\ggml.c:5758: ggml_can_mul_mat(a, b)` with an exit.

Originally posted by @alxspiker in https://github.com/su77ungr/CASALIOY/issues/27#issuecomment-1546867344

startLLM.py seems to be working fine and weirdly seems very fast on latest pip packages.

alxspiker commented 1 year ago

29 Should fix this!

alxspiker commented 1 year ago

Seems to be fixed on my end. Doing lots of queries in a row.

Queries without incident: 10