search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
23.17k stars 1.35k forks source link

bug: The entire LLama family is not working #3915

Open SimplyProgrammer opened 4 days ago

SimplyProgrammer commented 4 days ago

Jan version

0.5.7

Describe the Bug

The issue is de facto identical to the one I posted couple of months ago, only problem is that now all of the LLama models are refusing to start with oftentimes no reason given at all... The only notable difference is that now it is happening after updating from 0.5.6->0.5.7. In 0.5.6, it worked without problems.

Steps to Reproduce

The video pretty much describes it perfectly, this was right after I downloaded the models from Jan's Model Explorer...

https://github.com/user-attachments/assets/749cc0d8-0412-428d-be28-59748c8378ca

Screenshots / Logs

I have downloaded the cortex Apis in settings that LLamas are dependent on, as always after the Jan update, the only problem is that this time it did not really do much :/ image

If we are speaking about logs, I would love to provide them, the only problem is that my /jan/logs/ folder is empty, like there is nothing present even after performing the actions shown in the video...

I have the same hw. PC as mentioned in the old issue post.

What is your OS?

imtuyethan commented 3 days ago

Hi, thanks for reporting this! Quick check, were you be able to run any other models or it's just Llama? @SimplyProgrammer. Can you give more details on your device specs, that would help a lot since i can't reproduce this bug from our Window machines.

SimplyProgrammer commented 3 days ago

Hi. Yes I have tried installing and using also "Qwen2.5 14B Instruct Q4" and it is not starting either, so the issue does not seem to be specific to Llama family of models... I can try a specific model if you would like to... I have also tried to disable antivirus (Avast) but it did not help... Also my specs are: Full system and CPU info: image 64GB of DDR4 RTX 3080 Ti with 12GB of VRAM

louis-jan commented 3 days ago

Hi @SimplyProgrammer could you please help check the engines/[win-cuda] in the Jan data folder and share the file list here. There should be a couple of dll files.

SimplyProgrammer commented 2 days ago

I am afraid that I have no such a file... This is ~/jan: image

And this is the Jan installation: image locals and resources does not seem to be related...