search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
20.82k stars 1.19k forks source link

bug: Inference Engine Process Fails to Terminate After Closing Jan #3060

Open Van-QA opened 2 weeks ago

Van-QA commented 2 weeks ago

https://discord.com/channels/1107178041848909847/1252601327255683145

Describe the bug The inference engine process associated with Jan does not terminate even after Jan is closed, resulting in excessive VRAM usage.

To Reproduce

  1. Run Jan
  2. Perform inference
  3. Close Jan
  4. Observe that the inference engine process is still running

Expected behavior Upon closing Jan, the inference engine process should also terminate, releasing the used VRAM.

Screenshots image

Hardware: 2024-06-18T14:13:50.264Z [SPECS]::0, 8188, NVIDIA GeForce RTX 4070 Laptop GPU 2024-06-18T14:13:50.264Z [SPECS]::OS Platform: linux 2024-06-18T14:13:50.265Z [APP]::{"notify":true,"run_mode":"gpu","nvidia_driver":{"exist":true,"version":"550.90.07"},"cuda":{"exist":true,"version":"12"},"gpus":[{"id":"0","vram":"8188","name":"NVIDIA GeForce RTX 4070 Laptop GPU","arch":"ada"}],"gpu_highest_vram":"0","gpus_in_use":["0"],"is_initial":false}

Additional context The user has tried killing the process manually, but the issue persists. The logs show that the app is trying to kill the cortex-cpp server before it exits, suggesting that the app process may be being killed before the cortex-cpp process can be terminated or that a stronger signal than SIGTERM may be required to kill the cortex-cpp process.

Log message (9).txt

FabulousSoldier commented 1 week ago

I am encountering a similar issue on Arch Linux. After opening Jan, loading a model, and then closing the application, the process 'cortex-cpp' continues to run in the background. This causes a slowdown of the GPU, particularly noticeable when launching games. Each time I exit Jan, I need to manually terminate the 'cortex-cpp' process to resolve the problem.