Open madeepakkumar1 opened 1 year ago
@su77ungr Any idea how to fix it?
Getting this error while loading model gptj_model_load: invalid model file 'models\TheBloke\GPT4All-13B-snoozy-GGML\GPT4Al l-13B-snoozy.ggmlv3.q4_0.bin' (bad magic
did you check the identation. theres a space in the model path at least on your comment here GPT4All-13B-snoozy.ggmlv3.q4_0.bin
model get downloaded and placed at models folder
There is no space in .env
Not sure how that space is added !
Oh you are using gpt4all. so we need a gptj backend. not sure about the compatibility of that model. let me check this when im home again
@su77ungr , any idea if you have checked it?
.env
Generic
TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false
Ingestion
PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=5
Generation
MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp
MODEL_TYPE=GPT4All
MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin
MODEL_PATH=TheBloke/GPT4All-13B-snoozy-GGML/GPT4All-13B-snoozy.ggmlv3.q4_0.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 # Max total size of prompt+answer MODEL_MAX_TOKENS=500 # Max size of answer MODEL_STOP=[STOP] CHAIN_TYPE=betterstuff N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db N_FORWARD_DOCUMENTS=100 # How many documents to forward to the LLM, chosen among those retrieved N_GPU_LAYERS=4
Python version
python3.10.11
System
Windows 10
CASALIOY version
latest
Information
Related Components
Reproduction
$python casolioy/startLLM.py
Enter a query:
Expected behavior