janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
22.1k stars 1.27k forks source link

epic: Jan & Cortex support for AMD GPUs #3529

Open imtuyethan opened 2 weeks ago

imtuyethan commented 2 weeks ago

Background

Currently, Jan only supports NVIDIA GPUs for acceleration. Users with AMD GPUs, particularly those using eGPUs on Intel Macs, are unable to utilize their graphics hardware for acceleration in Jan. This limits the performance potential for a segment of Jan users who have invested in AMD GPU solutions.

Feature request

Proposed Implementation

For engineers to fill in

Additional Notes

Consider prioritizing support for popular AMD GPU models like the Vega series initially.

cracksauce commented 2 days ago

Commenting in support of this feature request