Open Stefan-Olt opened 1 month ago
I think nobody has yet tried to use 5500 XT, it would be nice to get it however supported as I think it's pretty power efficient gpu if there is just enough memory to run models.
It may take a while until I have time to check this one but I can try to make branch which would have similar type of changes that are for 1010 to be made for this one also.
Thanks, great to hear! I will see if I can run a 1010 build on it with setting a HSA_OVERRIDE_GFX_VERSION=10.1.0
environment variable, just like the AMD builds run on my RX 6600 with HSA_OVERRIDE_GFX_VERSION=10.3.0
Did you had time to test the gfx5500 yet by building it for gfx1010?
Not yet, I will test today. On the machine with the RX 5500 the build crashes, I assume not enough RAM. So I will have to compile it on another machine and then copy it (both run the same OS)
I have tested it now and it works. I also tried a real-world PyTorch application (openai-whisper), but it complained that librocm_smi64.so.5
could not be found. I solved that using ln -s librocm_smi64.so.2.8 librocm_smi64.so.5
. Not sure why PyTorch has the wrong version.
But RX 5500 seems to be a lot slower for whisper than my RX 6600 (both 8GB)
Nice. Would you have time to document under docs/ how to test the whisper?
I was looking for possibility to build the ollama. Have not tested it yet. https://github.com/ollama/ollama/blob/main/docs/development.md
I think they added to ollama the AMD support some months ago, not fully sure is some merges still pending.
Whisper is just using PyTorch, so by itself it's independent of cuda/rocm. I just tested it to check if a real-world application is running on a RX 5500. The entire test is basically:
source /opt/rock_sdk_611/bin/env_rocm.sh
pip3 install openai-whisper
whisper --model medium some_speech_audio_file.mp3
How exactly do you want me to document that? The medium model still fits in 8GB, the large model needs 12GB/16 GB of VRAM. And it needs ffmpeg to be installed
Ok, so whisper does not have any nvidia dependencies in itself when pip is used to install it?
Maybe there could then be own audio_whisper folder either under docs/examples/pytorch which has the pip install command and then the launch script examples for different model sizes.
Do you think that makes sense?
And if there is some more ways to tune it (kind of longer example with multiple steps) then it could go under docs/tutorials
Hello, I'm trying to compile for a RX 5500 on Linux Mint (Ubuntu 22.04 LTS), so I only selected gfx1012, unfortunately at some point I get the error that the GPU type is unknown:
This is the CMakeError.log:
Is this a bug or am I doing something wrong?