openvino and intel-tensorflow>2.7 are incompatible

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc

Apache License 2.0

6.69k stars 1.26k forks source link

openvino and intel-tensorflow>2.7 are incompatible #5522

Open MeouSker77 opened 2 years ago

MeouSker77 commented 2 years ago

When importing both openvino and intel-tensorflow > 2.7, the following code will raise an error: RuntimeError: dimension (-1) in node dim must be greater or equal to -1: at offset

import tensorflow
from bigdl.nano.openvino import OpenVINOModel
OpenVINOModel('tmp.xml')

Only importing openvino is ok:

from bigdl.nano.openvino import OpenVINOModel
OpenVINOModel('tmp.xml')

This error won't occur when using tcmalloc, but it will occur when using jemalloc or using neither.

jason-dai commented 2 years ago

@zhentaocc please take a look

zhentaocc commented 2 years ago

Opened an issue in OpenVINO https://github.com/openvinotoolkit/openvino/issues/12747.

jason-dai commented 2 years ago

Opened an issue in OpenVINO openvinotoolkit/openvino#12747.

Does it happen for previous versions?

zhentaocc commented 2 years ago

Opened an issue in OpenVINO openvinotoolkit/openvino#12747.

Does it happen for previous versions?

Tensorflow 2.8 is not compatible with lower version of OpenVINO. It will fail during the model conversion from TF to IR.
But if you mean TF version, OpenVINO 2022.1 works on TF 2.7.

Seems to be related to allocation... TF2.8+OpenVINO 2022.1+tcmalloc works.

jason-dai commented 2 years ago

Opened an issue in OpenVINO openvinotoolkit/openvino#12747.

Does it happen for previous versions?

Tensorflow 2.8 is not compatible with lower version of OpenVINO. It will fail during the model conversion from TF to IR. But if you mean TF version, OpenVINO 2022.1 works on TF 2.7.

Seems to be related to allocation... TF2.8+OpenVINO 2022.1+tcmalloc works.

How about default TF 2.8 & 2.9 with OpenVINO?

zhentaocc commented 2 years ago

https://github.com/intel-analytics/BigDL/runs/8032571472?check_suite_focus=true
replacing intel-tensorflow with tensorflow-cpu works.

zhentaocc commented 2 years ago

https://jira.devtools.intel.com/browse/TFDO-5498?orderby=priority+ASC%2C+updated+DESC issue for intel tensorflow

TheaperDeng commented 2 years ago

@MeouSker77 maybe we could add an issue to openvino team jira as well? (will send link offline)

MeouSker77 commented 2 years ago

@MeouSker77 maybe we could add an issue to openvino team jira as well? (will send link offline)

There is already an issue about this in openvino team jira

zhentaocc commented 2 years ago

@MeouSker77 maybe we could add an issue to openvino team jira as well? (will send link offline)

There is already an issue about this in openvino team jira

Do you have a ticket link for this?

MeouSker77 commented 2 years ago

Do you have a ticket link for this?

send the link offline