Open Kim-William opened 5 days ago
Hi @Kim-William -
Thanks for reporting the issue. Here you are getting this error AttributeError: Exception encountered when calling TFBertMainLayer.call().
because there in keras3(tensorflow 2.16+) the Input layer gives output of KerasTensor not tf.Tensor.
For convert KerasTensor to tf.Tensor you can need to use subclassing.
input_ids = Input(shape=(100,), dtype=tf.int32, name="input_ids")
attention_mask = Input(shape=(100,), dtype=tf.int32, name="attention_mask")
class BertLayer(keras.layers.Layer):
def __init__(self, bert_model):
super(BertLayer, self).__init__()
self.bert_model = bert_model
def call(self, inputs):
input_ids, attention_mask = inputs
bert_output= self.bert_model(input_ids=input_ids, attention_mask=attention_mask)
bert_output = bert_output.last_hidden_state
return bert_output
bert_model = TFBertModel.from_pretrained("bert-base-uncased")
bert_model.trainable = False
bert_layer = BertLayer(bert_model)
bert_output = bert_layer([input_ids, attention_mask])
Attached gist for the reference as well.
https://github.com/tensorflow/tensorflow/issues/77826
TensorFlow version: 2.17.0 Transformers version: 4.46.0.dev0 Keras version: 3.6.0
This problem can be solved by using TensorFlow version 2.11, but it is not solved because i'm trying to use Tensorflow version 2.17. The existing BERT model is implemented in version 2.17, but this also does not work in version 2.11 of Tensorflow...
Therefore, I would like to solve this problem and make the entire code work in version 2.17.