SciSharp / LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
https://scisharp.github.io/LLamaSharp
MIT License
2.71k stars 350 forks source link

Fix: Increase Vulkan detection timeout to 12 seconds #931

Closed LSXPrime closed 1 month ago

LSXPrime commented 1 month ago

The previous timeout of 1 second for detecting the Vulkan GPU using vulkaninfo --summary was insufficient on some systems, leading to the command timing out and the inference falling back to the CPU. This commit increases the timeout to 12 seconds to accommodate systems where vulkaninfo takes longer to complete.

This change addresses an issue where the Vulkan backend would frequently fall back to CPU inference due to the vulkaninfo command timing out before it could return the necessary information to detect the GPU. The increased timeout ensures that the command has sufficient time to complete on a wider range of systems, enabling the Vulkan backend to correctly detect and utilize the GPU for inference.

martindevans commented 1 month ago

Thanks for investigating and fixing this 😁