AMD Radeon RX 6900 XT on Linux fails with `--gpu amd` flag

johnshaughnessy commented 8 months ago

I have not been able to get llamafile --gpu amd working with an AMD Radeon RX 6900 XT on linux. The relevant line of the log seems to be:

llamafile: /usr/src/debug/hip-runtime-amd/clr-rocm-5.7.1/rocclr/os/os_posix.cpp:310: static void amd::Os::currentStackInfo(unsigned char**, size_t*): Assertion `Os::currentStackPtr() >= *base - *size && Os::currentStackPtr() < *base && "just checking"' failed.

full log

I found similar bug reports in other projects, so I suspect this is NOT a llamafile bug:

Instead, it seems that ROCm is not supported for my graphics card on linux:

  Name:                    gfx1030
  Marketing Name:          AMD Radeon RX 6900 XT

full rocminfo

Searching the AMD docs, I found:

I tried messing with the environment variable HSA_OVERRIDE_GFX_VERSION because I had seen that in some other issue reports, but did not have any luck.

In case it's helpful, I kept a log the steps I took when setting things up.

To summarize, I installed ROCm for Arch Linux, but it seems that my graphics card (Radeon RX 6900 XT) is not supported by ROCm on linux, so I cannot use the --gpu amd flag with llamafile.

If this is correct, then it is not a bug with llamafile. Still, I wanted to file this issue:

to ask if this seems correct,
to ask if there's anything else worth trying before giving up on my card,
to save anyone else the trouble of figuring this out,
to offer to make a note in the Gotchas section of the README.md.

AwesomeApple12 commented 8 months ago

Try ROCm 5.7.1, I think 6.0.0 is too new for your GPU. Also "export HSA_OVERRIDE_GFX_VERSION=10.3.0" should work just fine on your GPU. Try using the nightly pytorch.

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm5.7

hiepxanh commented 8 months ago

install windows and problem solve @johnshaughnessy

Mozilla-Ocho / llamafile

AMD Radeon RX 6900 XT on Linux fails with `--gpu amd` flag #214