search

openvinotoolkit / openvino_notebooks

📚 Jupyter notebook tutorials for OpenVINO™
Apache License 2.0
2.4k stars 809 forks source link

Loading the embedding model with NPU does not work #2364

Closed Nicogs43 closed 3 weeks ago

Nicogs43 commented 1 month ago

Describe the bug In the llm-rag-langchain.ipynb notebooks when I try to compile the embeddings model using the NPU I get an error back. Expected behavior It should be able to compile the model with NPU without returning errors Screenshots image

Installation instructions (Please mark the checkbox) [ ] I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks.

Environment information System information: Python executable: C:\Users\nico\openvino_env\Scripts\python.exe Pip version: 24.2 OpenVINO source: C:\Users\nico\openvino_env\lib\site-packages\openvino OpenVINO IE version: 2024.5.0-16610-41394bb0e6a OpenVINO environment activated: OK Jupyter kernel installed for openvino_env: NOT OK Python version: 3.10 OK OpenVINO pip package installed: OK OpenVINO import succeeds: OK OpenVINO development tools installed: NOT OK OpenVINO not installed globally: OK No broken requirements: NOT OK

The Python version in openvino_env does not match the openvino_env Jupyter kernel. This may not be an issue. If you experience issues, please follow the instructions in the README to reinstall the kernel.

OpenVINO development tools are not installed in this Python environment. Please follow the instructions in the README to install openvino-dev

pip check shows broken requirements: CompletedProcess(args=['pip', 'check'], returncode=1, stdout='ninja 1.11.1.1 is not supported on this platform\n')

The README.md file is located in the openvino_notebooks directory and at https://github.com/openvinotoolkit/openvino_notebooks Broken requirements are often harmless, but could cause issues. Additional context Add any other context about the problem here.

brmarkus commented 1 month ago

Have a look under "https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html#limitations". At least currently, for the NPU only static shapes are supported.

See also "https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html#dynamic-shapes" for the GPU plugin: "Currently, dynamic shape support for GPU is a preview feature and has the following limitations"

Nicogs43 commented 1 month ago

Have a look under "https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html#limitations". At least currently, for the NPU only static shapes are supported.

See also "https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html#dynamic-shapes" for the GPU plugin: "Currently, dynamic shape support for GPU is a preview feature and has the following limitations"

Thank you, I don't understand then the point of making a notebook with an example like this if it doesn't work. I don't understand then this part either: if USING_NPU: embedding.ov_model.reshape(1, 512) embedding.ov_model.compile()

brmarkus commented 1 month ago

Not all notebooks, not all models are for all sorts of accelerators...

This notebook "llm-rag-langchain" specifically seems to be made for CPU and (d)GPUs according to the comments inside.

Iffa-Intel commented 3 weeks ago

Closing issue, feel free to re-open or start a new issue if additional assistance is needed.