search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
21.16k stars 1.22k forks source link

bug: Load model failed with error TypeError: fetch failed, model : chimera-apex-7b IQ3M gguf #3102

Open asadnewbie opened 1 month ago

asadnewbie commented 1 month ago

Current behavior

after downloading the chimera-apex-7b IQ3-m.gguf model i choose it and then send message but no response( i waited and also check the cpu but no load on cpu). then in the settings i click on start model but it shows inactive.

Minimum reproduction step

  1. downloaded the model.
  2. send the message.
  3. no response.
  4. click on start model.
  5. still shows inactive.

Expected behavior

i expected that the model will work and give me any response.

Screenshots / Logs

2024-06-24T15:03:45.478Z [CORTEX]::Debug: Request to kill cortex 2024-06-24T15:03:45.482Z [CORTEX]::Debug: cortex process is terminated 2024-06-24T15:03:45.526Z [CORTEX]::CPU information - 2 2024-06-24T15:03:45.527Z [CORTEX]::Debug: Request to kill cortex 2024-06-24T15:03:45.531Z [CORTEX]::Debug: cortex process is terminated 2024-06-24T15:03:45.533Z [CORTEX]::Debug: Spawning cortex subprocess... 2024-06-24T15:03:45.534Z [CORTEX]::Debug: Spawn cortex at path: /home/asad/jan/extensions/@janhq/inference-cortex-extension/dist/bin/linux-cpu/cortex-cpp, and args: 1,127.0.0.1,3928 2024-06-24T15:03:45.534Z [APP]::/home/asad/jan/extensions/@janhq/inference-cortex-extension/dist/bin/linux-cpu 2024-06-24T15:03:45.648Z [CORTEX]::Debug: Loading model with params {"cpu_threads":2,"ctx_len":2048,"embedding":false,"prompt_template":"{system_message}\n### Instruction: {prompt}\n### Response:","llama_model_path":"/home/asad/jan/models/Chimera-Apex-7B.IQ3_M.gguf/Chimera-Apex-7B.IQ3_M.gguf","system_prompt":"","user_prompt":"\n### Instruction: ","ai_prompt":"\n### Response:","model":"Chimera-Apex-7B.IQ3_M.gguf","ngl":100} 2024-06-24T15:03:45.648Z [CORTEX]::Debug: cortex is ready 2024-06-24T15:03:45.656Z [CORTEX]::Debug: cortex exited with code: null 2024-06-24T15:03:46.565Z [CORTEX]::Error: Load model failed with error TypeError: fetch failed 2024-06-24T15:03:46.566Z [CORTEX]::Error: TypeError: fetch failed

Jan version

0.5.1 stable

In which operating systems have you tested?

Environment details

I have 8gb ram, processor: Intel i3-3120M, graphics: mesa Intel HD 4000( ivb gt2).

Van-QA commented 1 month ago

hi @asadnewbie, The issue is potentially related to https://github.com/janhq/jan/issues/3089, can you give us more info on your hardware?

Note that it's working on our Linux, so not a problem with the model image

Thank you