AttributeError: 'NoneType' object has no attribute 'shape'

[Kim-William]

https://github.com/tensorflow/tensorflow/issues/77826

TensorFlow version: 2.17.0 Transformers version: 4.46.0.dev0 Keras version: 3.6.0

This problem can be solved by using TensorFlow version 2.11, but it is not solved because i'm trying to use Tensorflow version 2.17. The existing BERT model is implemented in version 2.17, but this also does not work in version 2.11 of Tensorflow...

Therefore, I would like to solve this problem and make the entire code work in version 2.17.

[mehtamansi29]

Hi @Kim-William -

Thanks for reporting the issue. Here you are getting this error AttributeError: Exception encountered when calling TFBertMainLayer.call(). because there in keras3(tensorflow 2.16+) the Input layer gives output of KerasTensor not tf.Tensor.

For convert KerasTensor to tf.Tensor you can need to use subclassing.

input_ids = Input(shape=(100,), dtype=tf.int32, name="input_ids")
attention_mask = Input(shape=(100,), dtype=tf.int32, name="attention_mask")

class BertLayer(keras.layers.Layer):
    def __init__(self, bert_model):
        super(BertLayer, self).__init__()
        self.bert_model = bert_model

    def call(self, inputs):
        input_ids, attention_mask = inputs
        bert_output= self.bert_model(input_ids=input_ids, attention_mask=attention_mask)
        bert_output = bert_output.last_hidden_state
        return bert_output

bert_model = TFBertModel.from_pretrained("bert-base-uncased")
bert_model.trainable = False

bert_layer = BertLayer(bert_model)
bert_output = bert_layer([input_ids, attention_mask])

Attached gist for the reference as well.