search

bojone / bert4keras

keras implement of transformers for humans
https://kexue.fm/archives/6915
Apache License 2.0
5.37k stars 929 forks source link

利用OpenVINO进行模型转换时报错 #511

Closed dsx4602 closed 1 year ago

dsx4602 commented 1 year ago

提问时请尽可能提供如下信息：

基本信息

核心代码

加载bert模型，保存成SavedModel格式

import os
os.environ['TF_KERAS'] = '1'  # 必须使用tf.keras

import os
import numpy as np
from bert4keras.models import build_transformer_model
from bert4keras.tokenizers import Tokenizer
from bert4keras.snippets import to_array

# 加载预训练模型

model_dir = 'xxxx\\chinese_L-12_H-768_A-12'
config_path = os.path.join(model_dir, 'bert_config.json')
checkpoint_path =  os.path.join(model_dir, 'bert_model.ckpt')
dict_path =  os.path.join(model_dir, 'vocab.txt')

tokenizer = Tokenizer(dict_path, do_lower_case=True)  # 建立分词器
bert = build_transformer_model(
    config_path=config_path, checkpoint_path=checkpoint_path, with_mlm=False, return_keras_model=False
)  

import tensorflow as tf
tf.saved_model.save(bert.model, "./data/saved_models/original_bert/1")

输出信息

新开虚拟环境，根据 openvino官网安装教程安装OpenVINO™ Development Tools

pip install openvino-dev[tensorflow2,onnx]

安装的版本如下：

根据 openvino官网转换教程 进行模型转换（其中为实际的路径）

mo --saved_model_dir <parent_dir>\data\saved_models\original_bert\1 --input "input_token,input_segment"

报错说：

C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\ops\reshape.py:63: RuntimeWarning: overflow encountered in long_scalars
  num_of_output_elements *= x
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID (<class 'openvino.tools.mo.middle.ConvertGroupedStridedSlice.ConvertGroupedStridedSlice'>)": '<' not supported between instances of 'int' and 'NoneType'
[ ERROR ]  Traceback (most recent call last):
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\utils\class_registration.py", line 286, in apply_transform
    for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\middle\pattern_match.py", line 46, in for_graph_and_each_sub_graph_recursively
    func(graph)
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\middle\ConvertGroupedStridedSlice.py", line 92, in find_and_replace_pattern
    sorted_out_nodes = sorted(out_nodes, key=lambda n: list(n.slices))
TypeError: '<' not supported between instances of 'int' and 'NoneType'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\main.py", line 50, in main
    ngraph_function = convert_model(**argv)
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\convert.py", line 47, in convert_model
    return _convert(**args)
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\convert_impl.py", line 937, in _convert
    raise e.with_traceback(None)
Exception: Exception occurred during running replacer "REPLACEMENT_ID (<class 'openvino.tools.mo.middle.ConvertGroupedStridedSlice.ConvertGroupedStridedSlice'>)": '<' not supported between instances of 'int' and 'NoneType'

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------
[ INFO ] You can also try to use new TensorFlow Frontend (preview feature as of 2022.3) by adding `--use_new_frontend` option into Model Optimizer command-line.
Find more information about new TensorFlow Frontend at https://docs.openvino.ai/latest/openvino_docs_MO_DG_TensorFlow_Frontend.html

自我尝试

（1）尝试将转换命令改为如下所示以指定输入shape：

mo --saved_model_dir <parent_dir>\data\saved_models\original_bert\1 --input "input_token,input_segment" --input_shape [1,25],[1,25]

报错信息如下（似乎是int32和int64做了concat导致了报错）：

[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Check 'element::Type::merge(inputs_et, inputs_et, get_input_element_type(i))' failed at C:\Jenkins\workspace\private-ci\ie\build-windows-vs2019\b\repos\openvino\src\core\src\op\concat.cpp:43:
While validating node 'v0::Concat Concat_44 (StatefulPartitionedCall/model/Embedding-Position/strided_slice_3/stack_1/Unsqueeze[0]:i32[1], StatefulPartitionedCall/model/Embedding-Position/strided_slice_3/stack_1/Unsqueeze4478[0]:i32[1], StatefulPartitionedCall/model/Embedding-Position/strided_slice_3/extend_end_const38340251899[0]:i64[1]) -> ()' with friendly_name 'Concat_44':
Argument element types are inconsistent.

[ ERROR ]  Traceback (most recent call last):
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\main.py", line 50, in main
    ngraph_function = convert_model(**argv)
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\convert.py", line 47, in convert_model
    return _convert(**args)
  File "C:\Users\XXX\.conda\envs\python37_tf_test\lib\site-packages\openvino\tools\mo\convert_impl.py", line 937, in _convert
    raise e.with_traceback(None)
RuntimeError: Check 'element::Type::merge(inputs_et, inputs_et, get_input_element_type(i))' failed at C:\Jenkins\workspace\private-ci\ie\build-windows-vs2019\b\repos\openvino\src\core\src\op\concat.cpp:43:
While validating node 'v0::Concat Concat_44 (StatefulPartitionedCall/model/Embedding-Position/strided_slice_3/stack_1/Unsqueeze[0]:i32[1], StatefulPartitionedCall/model/Embedding-Position/strided_slice_3/stack_1/Unsqueeze4478[0]:i32[1], StatefulPartitionedCall/model/Embedding-Position/strided_slice_3/extend_end_const38340251899[0]:i64[1]) -> ()' with friendly_name 'Concat_44':
Argument element types are inconsistent.

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------
[ INFO ] You can also try to use new TensorFlow Frontend (preview feature as of 2022.3) by adding `--use_new_frontend` option into Model Optimizer command-line.
Find more information about new TensorFlow Frontend at https://docs.openvino.ai/latest/openvino_docs_MO_DG_TensorFlow_Frontend.html

（2）尝试使用最新版bert4keras代码（git上2023.3.8的代码）生成SavedModel，继续运行转换命令：

mo --saved_model_dir <parent_dir>\data\saved_models\original_bert\1 --input "input_token,input_segment" --input_shape [1,25],[1,25]

仍然报错，看上去仍然是int32和int64的问题：

[ ERROR ]  Cannot infer shapes or values for node "StatefulPartitionedCall/model_2/Transformer-0-MultiHeadSelfAttention/BroadcastArgs".
[ ERROR ]  Input 0 of node StatefulPartitionedCall/model_2/Transformer-0-MultiHeadSelfAttention/BroadcastArgs was passed int64 from StatefulPartitionedCall/model_2/Transformer-0-MultiHeadSelfAttention/Shape_3_port_0_ie_placeholder:0 incompatible with expected int32.
[ ERROR ]
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x000001A9F2742798>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'openvino.tools.mo.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/model_2/Transformer-0-MultiHeadSelfAttention/BroadcastArgs" node.
 For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvino.ai/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)
[ INFO ] You can also try to use new TensorFlow Frontend (preview feature as of 2022.3) by adding `--use_new_frontend` option into Model Optimizer command-line.
Find more information about new TensorFlow Frontend at https://docs.openvino.ai/latest/openvino_docs_MO_DG_TensorFlow_Frontend.html
bojone commented 1 year ago

你这个报错我看不出它跟bert4keras的联系呀