Closed rus-mihai closed 1 year ago
just merged #80. Hope this resolves this
unfortunately it doesn;t solve it , i just pulled latest and retried. But now I have more details :
Fatal Python error: Illegal instruction
Current thread 0x00007fffff75d4c0 (most recent call first):
File "/srv/CASALIOY/.venv/lib/python3.11/site-packages/llama_cpp/llama_cpp.py", line 154 in llama_context_default_params
File "/srv/CASALIOY/.venv/lib/python3.11/site-packages/llama_cpp/llama.py", line 130 in init
File "/srv/CASALIOY/.venv/lib/python3.11/site-packages/langchain/llms/llamacpp.py", line 133 in validate_environment
File "/srv/CASALIOY/casalioy/startLLM.py", line 57 in init
File "/srv/CASALIOY/casalioy/startLLM.py", line 123 in main
File "/srv/CASALIOY/casalioy/startLLM.py", line 135 in
Jap ok. I think this is caused by the slim base
Might give this a try:
Most likely not caused by slim, since it works fine on my end :P
Most likely not caused by slim, since it works fine on my end :P
You are running it also on a M1 (arm) MAC OS ?
PS : i'm building the image now. will let you know
I actually have it working now. Not sure because of the python base image or, because i built the image locally. Thank you
Update: If I pull from ~su77ungr/casalioy:latest~ su77ungr/casalioy:stable I get the same error as you do, but if I built it from my computer it works fine. Looks like you have the same behavior. Maybe it's an issue of architecture detected during the compilation ?
I think we should reopen this
Oh we should purge :latest it's four days old now
Sorry I meant stable
Nah, that's my fault. We should bring this up to latest. Currently stable works besides #83? Then we should keep this image and bring the :latest up too speed
Update: If I pull from ~su77ungr/casalioy:latest~ su77ungr/casalioy:stable I get the same error as you do, but if I built it from my computer it works fine. Looks like you have the same behavior. Maybe it's an issue of architecture detected during the compilation ?
I think we should reopen this
That's so weird I tested several VMs and no problem. Also I have no idea about MacOS never used it in life.
@hippalectryon-0 why is the docker image suddenly 750MB again?
You tell me :P did you change something ?
let's switch to #87
.env
Generic
TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=true
Ingestion
PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50
Generation
MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 # Max total size of prompt+answer MODEL_MAX_TOKENS=256 # Max size of answer MODEL_STOP=[STOP] CHAIN_TYPE=betterstuff N_RETRIEVE_DOCUMENTS=100 # How many documents to retrieve from the db N_FORWARD_DOCUMENTS=100 # How many documents to forward to the LLM, chosen among those retrieved N_GPU_LAYERS=4
Python version
Python 3.11.3
System
Debian GNU/Linux 11 (bullseye) (DOCKER container)
CASALIOY version
su77ungr/casalioy:stable
Information
Related Components
Reproduction
Steps to reproduce (on Mac m1):
docker pull su77ungr/casalioy:stable docker run -it su77ungr/casalioy:stable /bin/bash
python casalioy/ingest.py
Downloading model sentence-transformers/all-MiniLM-L6-v2 from HF Downloading (…)_Pooling/config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 190/190 [00:00<00:00, 684kB/s] Downloading (…)55de9125/config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 612/612 [00:00<00:00, 3.70MB/s] Downloading (…)125/data_config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 39.3k/39.3k [00:00<00:00, 4.90MB/s] Downloading (…)ce_transformers.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 116/116 [00:00<00:00, 626kB/s] Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 641kB/s] Downloading (…)nce_bert_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 53.0/53.0 [00:00<00:00, 309kB/s] Downloading (…)5de9125/modules.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 349/349 [00:00<00:00, 2.15MB/s] Downloading (…)okenizer_config.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 350/350 [00:00<00:00, 1.64MB/s] Downloading (…)e9125/tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 466k/466k [00:00<00:00, 1.31MB/s] Downloading pytorch_model.bin: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 90.9M/90.9M [00:18<00:00, 4.96MB/s] Fetching 10 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:24<00:00, 2.41s/it] Downloading model eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin from HF Downloading ggml-vic7b-q5_1.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.06G/5.06G [11:31<00:00, 7.31MB/s] Fetching 1 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [11:37<00:00, 697.51s/it] The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling
transformers.utils.move_cache(). 0it [00:00, ?it/s] Scanning files Processing state_of_the_union.txt Processing 90 chunks Creating a new collection, size=384 Saving 90 chunks Saved, the collection now holds 90 documents. Processed state_of_the_union.txt Processing sample.csv Processing 9 chunks Saving 9 chunks Saved, the collection now holds 99 documents. Processed sample.csv Processing shor.pdf Processing 22 chunks Saving 22 chunks Saved, the collection now holds 121 documents. Processed shor.pdf Processing Muscle Spasms Charley Horse MedlinePlus.html [nltk_data] Downloading package punkt to /root/nltk_data...===================> ] 3/ 7 eta [00:19] [nltk_data] Unzipping tokenizers/punkt.zip. 21 [nltk_data] Downloading package averaged_perceptron_tagger to 2 [nltk_data] /root/nltk_data... [nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip. Processing 15 chunks Saving 15 chunks Saved, the collection now holds 136 documents. Processed Muscle Spasms Charley Horse MedlinePlus.html Processing Easy_recipes.epub Processing 31 chunks Saving 31 chunks Saved, the collection now holds 167 documents. Processed Easy_recipes.epub Processing Constantinople.docx Processing 13 chunks Saving 13 chunks Saved, the collection now holds 179 documents. Processed Constantinople.docx Processing LLAMA Leveraging Object-Oriented Programming for Designing a Logging Framework-compressed.pdf Processing 14 chunks Saving 14 chunks Saved, the collection now holds 193 documents. Processed LLAMA Leveraging Object-Oriented Programming for Designing a Logging Framework-compressed.pdf 100.0% [==================================================================================================================================================================>] 7/ 7 eta [00:00] Done
root@6e62f96184c4:/srv/CASALIOY# python casalioy/startLLM.py found local model dir at models/sentence-transformers/all-MiniLM-L6-v2 found local model file at models/eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin
Illegal instruction
Expected behavior
I would expect to start the chatting