marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.
MIT License
1.76k stars 137 forks source link

core dumped / segmentation fault #176

Open lysa324 opened 8 months ago

lysa324 commented 8 months ago

Hi, I use debian 11, python 3.10, anaconda3 langchain==0.0.240 RAM = 16 Go swap = 10 Go SSD = 60 GO I try to use the simplest program with Ctransfors:

from langchain.llms import CTransformers
from langchain.chains import LLMChain
from langchain import PromptTemplate

prompt_template = """
You are an AI coding assistant and your task to solve the coding problems, and return coding snippets based on the
Query: {query}

You just return helpful answer and nothing else
Helpful Answer: 
"""

prompt = PromptTemplate(template=prompt_template, input_variables=['query'])

llm = CTransformers(model = "model/codellama-7b-instruct.ggmlv3.Q4_0.bin",
                    model_type = "llama",
                    max_new_tokens=512,
                    temperature=0.2
                    )

llm_chain = LLMChain(prompt=prompt, llm=llm)

llm_response = llm_chain.run({"query": "Write a python code to load a CSV file using pandas library"})

print(llm_response)

and i have "core dumped " error without other details, i tried a lot of things but nothig resolve my problem !

can you help me please thks

dipeshpaulsystango commented 3 months ago

Same issue, someone please help. It's also happening with pure cTransformers with .gguf models.