search

janhq / jan

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)
https://jan.ai/
GNU Affero General Public License v3.0
22.51k stars 1.3k forks source link

feat: ROCm support on Linux for dedicated AMD Radeon GPUs #2676

Closed tejasraman closed 1 month ago

tejasraman commented 5 months ago

Problem I use Jan on my Linux PC with an AMD dedicated (ROCm-supported) card. However, HIP/ROCm support for Jan is not present, so models are still run on the CPU.

Success Criteria In order for this proposal to be successful, full support for ROCm (and, optionally, AMD's legacy Polaris OpenCL implementation) should be added and be accessible through the same settings menu as CUDA acceleration.

metaspartan commented 5 months ago

Use Experimental settings in Jan to turn on Vulkan for your GPU, Jan uses Nitro and it uses Vulkan APIs currently instead of ROCm, but ROCm API support would be nice alt.

xtarlit commented 5 months ago

Is ROCm expected to perform better than the current implementation of Vulkan on Linux?

Van-QA commented 5 months ago

related to https://github.com/janhq/cortex/issues/323

eckartal commented 3 months ago

We received 10+ comments on AMD support yesterday. Is it possible to prioritize it?

imtuyethan commented 1 month ago

Closed as duplicated with https://github.com/janhq/cortex/issues/323