Feature Request: OpenVINO backend support request - Githubissues

ggerganov / llama.cpp

LLM inference in C/C++

MIT License

67.84k stars 9.73k forks source link

Feature Request: OpenVINO backend support request #9601

Closed aropb closed 6 days ago

aropb commented 1 month ago

Prerequisites

[X] I am running the latest code. Mention the version if possible as well.
[X] I carefully followed the README.md.
[X] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
[X] I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

OpenVINO backend support request: https://docs.openvino.ai/2024/index.html

Motivation

Rich possibilities of using CPU, GPU, VPU, NPU

Possible Implementation

No response

BarfingLemurs commented 1 month ago

Rich possibilities of using CPU, GPU, VPU, NPU

SYCL (Intel) is supported in llama.cpp

kylo5aby commented 1 month ago

Rich possibilities of using CPU, GPU, VPU, NPU

SYCL (Intel) is supported in llama.cpp

I think one benefit of supporting the OpenVINO backend is to enable Intel NPU

github-actions[bot] commented 6 days ago

This issue was closed because it has been inactive for 14 days since being marked as stale.