search

containers / podman-desktop-extension-ai-lab-playground-images

Apache License 2.0
1 stars 4 forks source link

feat: add vulkan images #22

Closed lstocchi closed 2 months ago

lstocchi commented 3 months ago

What does this PR do?

libkrun provider has been added to podman and soon we should have a way to start an inference server that is able to use gpu on a mac. This PR adds the creation of vulkan images so that, eventually, we can use them to start an inference server with gpu capabilities.

Screenshot / video of UI

N/A

What issues does this PR fix or reference?

N/A

How to test this PR?

  1. check that the image build correctly
lstocchi commented 3 months ago

The one from ai-lab-recipes uses library krunkit specific ?

It is primarily intended to be used on macOS with podman and krunkit but Sergio also said that it can be used on Linux with a VMM exposing a Venus-capable virtio-gpu.

jeffmaury commented 3 months ago

The one from ai-lab-recipes uses library krunkit specific ?

It is primarily intended to be used on macOS with podman and krunkit but Sergio also said that it can be used on Linux with a VMM exposing a Venus-capable virtio-gpu.

Did you check that performances with this images are the same compared to the benchmarks we did on MacOS/krunkit ?

lstocchi commented 3 months ago

Did you check that performances with this images are the same compared to the benchmarks we did on MacOS/krunkit ?

What do you mean? Didn't we use the image generated by the ai-lab-recipes team for the benchmark? I just copy/paste the containerfile so the result should be the same, no?

jeffmaury commented 3 months ago

Did you check that performances with this images are the same compared to the benchmarks we did on MacOS/krunkit ?

What do you mean? Didn't we use the image generated by the ai-lab-recipes team for the benchmark? I just copy/paste the containerfile so the result should be the same, no?

Does not seem to me as the ai-lab-recipe one contains libkrun specific libraries: see https://github.com/containers/ai-lab-recipes/blob/624155c699b7c2abc3c5878fba21ad848b23af25/model_servers/llamacpp_python/vulkan/arm64/Containerfile#L6

lstocchi commented 3 months ago

Does not seem to me as the ai-lab-recipe one contains libkrun specific libraries: see https://github.com/containers/ai-lab-recipes/blob/624155c699b7c2abc3c5878fba21ad848b23af25/model_servers/llamacpp_python/vulkan/arm64/Containerfile#L6

-> https://github.com/containers/podman-desktop-extension-ai-lab-playground-images/pull/22/files#diff-69e31da53e7e524501214e0e65d027964ba095c665858c15aa178fa24379d445R6