Update ollama to 0.1.28

[vrinek]

Tasklist

Builds

• [x] built default
• [x] built gpu
• [x] built rocm
• [x] built cuda
• [x] built cpu

Tests (with an AMD RX 7900 XTX and a Ryzen 9 5900X)

• [x] tested default (uses GPU)
• [x] tested gpu (uses GPU)
• [x] tested rocm (uses GPU)
• [x] tested cuda (uses CPU)
• [x] tested cpu (uses CPU)

[vrinek]

0.1.28 has an issue with its new models (starcoder2 and dolphincoder) that others are experiencing: https://github.com/ollama/ollama/issues/2953

[abysssol]

I'm considering archiving this repository since upstream nixpkgs is getting consistently updated now. Ollama 0.1.28 has been merged into nixpkgs master for a couple days now, and will probably be in nixpkgs-unstable within a week.

Is there any good reason to duplicate the ollama package for this flake? Is having a flake interface more convenient in some way than just using nixpkgs? Is waiting a week after release really that long? Any thoughts/comments?

[vrinek]

I recall I had problems getting my AMD GPU recognized by ollama as packaged in nixpkgs, but no problems with your flake. I'll give it another shot in case that has been fixed.

Apart from that, no, I don't see much of a benefit maintaining this as an independent flake.

Thanks for your work!

[abysssol]

Please do try the version in nixpkgs and tell me if you have any problems. If you do, the best thing to do is probably to open an issue on the nixos/nixpkgs repo and @ me in it.

You should be able to use ollama by putting this in your nixos configuration:

services.ollama.enable = true;
services.ollama.acceleration = "rocm";

Don't forget to update your flake input or channel to get the new version of ollama:

sudo nix flake lock --update-input nixos

sudo nix-channel --update nixos