tensorflow BERT分类模型转换报3 OPs are not supported yet

[vForce825]

使用bert4keras训练的BERT分类模型经过转换后报错提示有3个OP不支持转换： paddle.version = 1.7.2 Now translating model from tensorflow to paddle.

Shape[now is [-1, -1]] for input tensor[tensor name: "Input-Token"] not support yet Use your keyboard type the shape of input tensor below :) Shape of Input(e.g. None,224,224,3): None,256

Shape[now is [-1, -1]] for input tensor[tensor name: "Input-Segment"] not support yet Use your keyboard type the shape of input tensor below :) Shape of Input(e.g. None,224,224,3): None,256 Total nodes: 3583 Converting node 3583 ... ========= 3 OPs are not supported yet =========== ========== Unpack ============ ========== Square ============ ========== NotEqual ============

[jiangjiajun]

使用bert4keras训练的BERT分类模型经过转换后报错提示有3个OP不支持转换： paddle.version = 1.7.2 Now translating model from tensorflow to paddle.

Shape[now is [-1, -1]] for input tensor[tensor name: "Input-Token"] not support yet Use your keyboard type the shape of input tensor below :) Shape of Input(e.g. None,224,224,3): None,256

Shape[now is [-1, -1]] for input tensor[tensor name: "Input-Segment"] not support yet Use your keyboard type the shape of input tensor below :) Shape of Input(e.g. None,224,224,3): None,256 Total nodes: 3583 Converting node 3583 ... ========= 3 OPs are not supported yet =========== ========== Unpack ============ ========== Square ============ ========== NotEqual ============

方便的分享一下你的模型，便于我们debug

[vForce825]

使用bert4keras训练的BERT分类模型经过转换后报错提示有3个OP不支持转换： paddle.version = 1.7.2 Now translating model from tensorflow to paddle. Shape[now is [-1, -1]] for input tensor[tensor name: "Input-Token"] not support yet Use your keyboard type the shape of input tensor below :) Shape of Input(e.g. None,224,224,3): None,256 Shape[now is [-1, -1]] for input tensor[tensor name: "Input-Segment"] not support yet Use your keyboard type the shape of input tensor below :) Shape of Input(e.g. None,224,224,3): None,256 Total nodes: 3583 Converting node 3583 ... ========= 3 OPs are not supported yet =========== ========== Unpack ============ ========== Square ============ ========== NotEqual ============

方便的分享一下你的模型，便于我们debug

import tensorflow as tf import os from keras.backend.tensorflow_backend import set_session import numpy as np from bert4keras.backend import keras, K from bert4keras.models import build_transformer_model from bert4keras.tokenizers import Tokenizer, load_vocab from bert4keras.optimizers import Adam from bert4keras.snippets import sequence_padding, DataGenerator from keras.layers import * from keras.models import Model from sklearn.metrics import classification_report from sklearn.model_selection import train_test_split from tqdm import tqdm import pandas as pd from ast import literal_eval from keras.utils import to_categorical import re maxlen = 512 epochs = 5 batch_size = 32 bert_layers = 12 learing_rate = 1e-5 config_path = './google_bert/ernie_1.0_base_chinese/bert_config.json' checkpoint_path = './google_bert/ernie_1.0_base_chinese/bert_model.ckpt' dict_path = './google_bert/ernie_1.0_base_chinese/vocab.txt' token_dict, keep_tokens = load_vocab( dict_path=dict_path, simplified=True, startswith=['[PAD]', '[UNK]', '[CLS]', '[SEP]'], ) tokenizer = Tokenizer(token_dict, do_lower_case=True) model = build_transformer_model( config_path, checkpoint_path, )

for layer in model.layers: if 'Transformer-%s' % (bert_layers - 1) not in layer.name\ and 'Transformer-%s' % (bert_layers - 2) not in layer.name\ and 'Transformer-%s' % (bert_layers - 3) not in layer.name\ and 'Transformer-%s' % (bert_layers - 4) not in layer.name: layer.trainable = False

output_layers = [] for i in range(1, 5): output_layer = 'Transformer-%s-FeedForward-Norm' % (bert_layers - i) output = model.get_layer(output_layer).output output = Lambda(lambda x: x[:, 0], name='CLS-token-%s' % i)(output) output_layers.append(output) concat = Concatenate(name='concat')(output_layers) output = Dense(3, activation='softmax', name='final_dense')(concat) model = Model(model.input, output) model.summary() model.compile(loss='categorical_crossentropy', optimizer=Adam(learing_rate), metrics=['accuracy'])

[jiangjiajun]

麻烦直接上传一下你给X2Paddle转换的pb模型文件

[vForce825]

麻烦直接上传一下你给X2Paddle转换的pb模型文件

链接: https://pan.baidu.com/s/1o59yb5d8NYn4cn3A9DVF6Q 密码: cwr0

[jiangjiajun]

你好，我已经提了新的PR，修复了Concat OP的问题，目前可以支持你上面的模型，请从github下载最新的x2paddle代码进行安装后，再行转换

[vForce825]

你好，我已经提了新的PR，修复了Concat OP的问题，目前可以支持你上面的模型，请从github下载最新的x2paddle代码进行安装后，再行转换

更新后尝试转换发生了新的错误： Error: ShapeError: broadcast dimension mismatch. Operands could not be broadcast together with the shape of X = [-1, 256, 768] and the shape of Y = [-1, 254, 768]. Received [256] in X is not equal to [254] in Y at (/paddle/paddle/fluid/operators/elementwise/elementwise_op_function.h:145) [operator < elementwise_add > error] 给定的输入tensor shape为：None,256

[jiangjiajun]

我这边测试可正常转换