search

containers / podman-desktop-extension-ai-lab-playground-images

Apache License 2.0
1 stars 4 forks source link

chore: update to llama cpp python server 0.2.90 #53

Closed jeffmaury closed 6 days ago

jeffmaury commented 1 week ago

What does this PR do?

Switch llama-cpp-python to 0.2.90 and update Vulkan as compilation changed

Screenshot / video of UI

N/A

What issues does this PR fix or reference?

N/A

How to test this PR?

N/A

jeffmaury commented 1 week ago

is the jeffmaury repository expected in the final PR ?

For the time being, yes. The newest llama-cpp-python requires for Vulkan compilation a new package that is not in EPEL9. So I mimic what Sergio did for libkrun and created those packages in Fedora COPR. Will check if we can add them in EPEL9

benoitf commented 1 week ago

I was just wondering if it could not be part of the Dockerfile ? so it's maintained in the Dockerfile

jeffmaury commented 1 week ago

I was just wondering if it could not be part of the Dockerfile ? so it's maintained in the Dockerfile

That's what I did initially but that causes the build to take several hours (ARM emulation ?)

benoitf commented 6 days ago

ok need to wait the end of the year then https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/#get-started-using-arm-hosted-runners-today

there will be arm64 linux runners