Open Gutifarra opened 1 month ago
I'm just a random internet stranger, but i've recently done a PR to update the ollama and open-webui ujust commands in bluefin, and once that's in, i'm planning a PR to bring the just file over to bazzite too. https://github.com/ublue-os/bluefin/pull/1588 But FYI for now, if you just use the just file from the bluefin/just/bluefin-tools.just file on bazzite (just --justfile bluefin-tools.just ollama" it will work if you want to get it up and running right now on your machine.
When we're done in bluefin let's just move it to the config repo, that way it's in both and centralized.
@Gutifarra would you be willing to do a little testing on this PR? https://github.com/ublue-os/bluefin/pull/1588
maybe just try out rhe recipes from the justfile as you have an AMD GPU?
I was able to run it with docker container, using podman. Ill share scripts that I used in a bit.
[Update]
sudo podman run -it --name ollama --replace -p 127.0.0.1:11434:11434 -v ollama:/root/.ollama --device /dev/kfd --device /dev/dri docker.io/ollama/ollama:rocm
You would need to run it in sudo
, not sure why but even after adding the user to render
group it doesn't work.
Looks like it was SELinux,
sudo setsebool -P container_use_devices=true
running this allows podman to access the GPU devices without root.
Describe the bug
First off, thanks for your support and for the work done with bazzite.
I've been looking through various posts on the Discourse which seem to indicate that:
ujust
install script included in Bazzite to install ollama.This should mean that the
ujust ollama
mentioned in eg. this post should work, but for some reason it is not present on my system:Not sure if this is expected behavior, if there is some other way of installing ollama (using the script in the ollama site does not support ROCm), or if this is some bug in my setup.
Thanks for any support you can give and if I've made any mistakes.
What did you expect to happen?
Ollama to be installed with ROCm support.
Output of
rpm-ostree status
Hardware
AMD 5600X 32GB RAM AMD 6700XT
Extra information or context
No response