I'm using cTransformers with streamlit to provide a UI to an LLM. After running the model once on my machine and closing the application and trying to start up again, I get repeated segmentation faults and occasionally the following error:
I specified threads=-1 in the argument to the LLM which seemed to briefly fix the issue - but now it's still occuring.
I'm not sure if this is a cTransformers specific issue, related to ggml or even streamlit but thought it was worth posting as I can't seem to fix it at the moment. I'm on an EC2 instance running ubuntu 20.04, using TheBloke's quantized version of Mistral.
I'm using
cTransformers
withstreamlit
to provide a UI to an LLM. After running the model once on my machine and closing the application and trying to start up again, I get repeated segmentation faults and occasionally the following error:I specified
threads=-1
in the argument to the LLM which seemed to briefly fix the issue - but now it's still occuring.I'm not sure if this is a cTransformers specific issue, related to
ggml
or evenstreamlit
but thought it was worth posting as I can't seem to fix it at the moment. I'm on an EC2 instance running ubuntu 20.04, usingTheBloke
's quantized version ofMistral
.