在tensorflow1.14下使用bert4keras报错

[cdj0311]

问题

使用tensorflow api时bert4keras报错，错误代码在tf.layers.dense这个api，如果不使用这个api，直接输出bert的向量没有问题。

基本信息

• 你使用的Python版本: 3.6
• 你使用的Tensorflow版本: 1.14
• 你使用的Keras版本: 2.3.1
• 你使用的bert4keras版本: 最新
• 你使用纯keras还是tf.keras: tf.keras
• 你加载的预训练模型: chinese_roberta_wwm_ext_L-12_H-768_A-12

核心代码

# coding: utf-8
import os
import numpy as np
import tensorflow as tf
from bert4keras.tokenizers import Tokenizer
from bert4keras.snippets import sequence_padding
from bert4keras.models import build_transformer_model

os.environ["TF_KERAS"] = "1"

bert_dir = "./chinese_roberta_wwm_ext_L-12_H-768_A-12"
bert_config = os.path.join(bert_dir, 'bert_config.json')
bert_ckpt = os.path.join(bert_dir, "bert_model.ckpt")
tokenizer = Tokenizer(os.path.join(bert_dir, "vocab.txt"), do_lower_case=True)

max_len = 10

def build_model():
    token = tf.placeholder(shape=(None, max_len), dtype=tf.int64, name="token")
    segment = tf.placeholder(shape=(None, max_len), dtype=tf.int64,  name="segment")

    bert = build_transformer_model(
        config_path=bert_config,
        checkpoint_path=bert_ckpt,
        model='bert',
        num_hidden_layers=1,
        return_keras_model=True
    )
    bert_encoder = bert([token, segment])
    bert_encoder = bert_encoder[:,0]
    bert_encoder= tf.layers.dense(bert_encoder, units=1)
    return bert_encoder

text = "测试"
token_id, segment_id = tokenizer.encode(text, maxlen=max_len)
token_id = token_id[:max_len] + [0]*(max_len - len(token_id))
segment_id = segment_id[:max_len] + [0]*(max_len - len(segment_id))
feed_dict = {"token:0": np.array([token_id]), "segment:0": np.array([segment_id])}

with tf.Session() as sess:
    tf.global_variables_initializer()
    model = build_model()
    print(sess.run(model, feed_dict=feed_dict))

输出信息

Traceback (most recent call last):
  File "test_estimator_bert.py", line 49, in <module>
    model = build_model()
  File "test_estimator_bert.py", line 36, in build_model
    title_bert = tf.layers.dense(bert_encoder, units=1)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/layers/core.py", line 188, in dense
    return layer.apply(inputs)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1479, in apply
    return self.__call__(inputs, *args, **kwargs)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 537, in __call__
    outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 561, in __call__
    base_layer_utils.create_keras_history(inputs)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 200, in create_keras_history
    _, created_layers = _create_keras_history_helper(tensors, set(), [])
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 253, in _create_keras_history_helper
    layer_inputs, op.outputs)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1795, in _add_inbound_node
    input_tensors)
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 515, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 515, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/data/anaconda3/envs/py3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1794, in <lambda>
    inbound_layers = nest.map_structure(lambda t: t._keras_history.layer,
AttributeError: 'tuple' object has no attribute 'layer'

自我尝试

换了几个tf版本都一样的错误

[bojone]

os.environ["TF_KERAS"] = "1"放到最开头。

环境变量是给库读取的，要在导入bert4keras之前设置好环境变量。

[cdj0311]

os.environ["TF_KERAS"] = "1"放到最开头。

环境变量是给库读取的，要在导入bert4keras之前设置好环境变量。

感谢苏神！