The short of it is that I tried using a Meta-llama3 70b image at some point, and that specific image seemed to corrupt the cache that is there to where nothing else loaded anymore and would end up in these various Illegal Instruction, or segfault at prompt scenarios. Might be worth looking at.
After deleting that folder and running a 'previously working' image again, it now transfered from an immediate Illegal Instruction, to a segfault, so I'm going to go down the path of looking at other caches, do you know what all gets deployed to user home?. So I think there might need to be some validation of what is there versus what is expected to be there.
Rel images, I went from mixtral 7B to Meta 70B and then everything else no longer worked right.
Meta-Llama-3-70B-Instruct.Q5_K_M.llamafile
mixtral-8x7b-instruct-v0.1.Q5_K_M.llamafile
Mixtral-8x22B-Instruct-v0.1.Q4_0.llamafile
llava-v1.5-7b-q4-server.llamafile
Relevant discussion here: https://www.reddit.com/r/LocalLLaMA/comments/1cvuf1z/fun_times_running_models_cpu_only_qa_hopefully/
The short of it is that I tried using a Meta-llama3 70b image at some point, and that specific image seemed to corrupt the cache that is there to where nothing else loaded anymore and would end up in these various Illegal Instruction, or segfault at prompt scenarios. Might be worth looking at.
After deleting that folder and running a 'previously working' image again, it now transfered from an immediate Illegal Instruction, to a segfault, so I'm going to go down the path of looking at other caches, do you know what all gets deployed to user home?. So I think there might need to be some validation of what is there versus what is expected to be there.
Rel images, I went from mixtral 7B to Meta 70B and then everything else no longer worked right. Meta-Llama-3-70B-Instruct.Q5_K_M.llamafile
mixtral-8x7b-instruct-v0.1.Q5_K_M.llamafile Mixtral-8x22B-Instruct-v0.1.Q4_0.llamafile llava-v1.5-7b-q4-server.llamafile