search

openvino-dev-samples / Qwen2.openvino

This sample shows how to deploy Qwen2 using OpenVINO
32 stars 3 forks source link

可以转为Intel NPU 吗? #5

Open AlbertG123 opened 4 months ago

AlbertG123 commented 4 months ago

Intel(R) Core(TM) Ultra 5 125H 1.20 GHz Thinkpad t14p Gen2 可以跑在这个NPU 上吗？ 看你的guide 只写了 GPU

openvino-dev-samples commented 4 months ago

很抱歉，目前不支持NPU

AlbertG123 commented 4 months ago

Ethan 好: 这个是否支持NPU 是什么决定啊？

   openvino 主页中提到这些model 应该都能转为NPU 的吧?

[cid:a0897084-65c7-4fb1-9801-c20dceafac13]

From: Ethan Yang @.> Sent: Thursday, June 20, 2024 11:49 AM To: OpenVINO-dev-contest/Qwen2.openvino @.> Cc: Albert Gao @.>; Author @.> Subject: [External] Re: [OpenVINO-dev-contest/Qwen2.openvino] 可以转为Intel NPU 吗? (Issue #5)

很抱歉，目前不支持NPU

― Reply to this email directly, view it on GitHubhttps://github.com/OpenVINO-dev-contest/Qwen2.openvino/issues/5#issuecomment-2179753657, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A2B3U4TAIF7VVYG62TDS6M3ZIJGMTAVCNFSM6AAAAABJTDRAVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZZG42TGNRVG4. You are receiving this because you authored the thread.Message ID: @.***>

openvino-dev-samples commented 4 months ago

目前NPU无法支持动态输入，所以跑LLM需要通过padding或是将原始LLM拆成两个来跑，造成的结果就是性能不理想。

walterdreamlin commented 1 month ago

https://github.com/openvinotoolkit/openvino/blob/master/docs/articles_en/learn-openvino/llm_inference_guide/genai-guide-npu.rst

为何他这可以用NPU？