search

rustformers / llm

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models
https://docs.rs/llm/latest/llm/
Apache License 2.0
6.07k stars 355 forks source link

AMD ROCm support with HIPBLAS #415

Open xangelix opened 1 year ago

xangelix commented 1 year ago

https://github.com/ggerganov/llama.cpp/pull/1087 was merged recently and should make its way into GGML shortly. I think (hopefully) it should require only minimal changes on this side.

xangelix commented 1 year ago

related to https://github.com/ggerganov/ggml/issues/472

LLukas22 commented 1 year ago

rustformers uses llama.cpp as it's ggml source, feel free to create an PR including this change, seams like you only need to adjust the build.rs of the ggml-sys create. I wont be able to test this as i don't own a amd gpu.