Paddings must be non-negative

[brunopistone]

I'm getting the following error when I'm trying to train a model for generating the next word in a sentence:

Paddings must be non-negative for 'Pad' (op: 'Pad') with input shapes: [1071,768], [2,2] and with computed input tensors: input[1] = <[0 -559][0 0]>.

The pre-trained model used is multi_cased_L-12_H-768_A-12.

This is the code:

bert_layer = bert.BertModelLayer.from_params(self.bert_params, name="bert")
bert_params = bert.params_from_pretrained_ckpt(model_path)
tokenizer = bert.bert_tokenization.FullTokenizer(os.path.join(model_path, "vocab.txt"), do_lower)

model = tf.keras.Sequential([
        tf.keras.layers.Input(shape=(constants.MAX_SEQ_LENGTH - 1,), dtype='int32', name='input_ids'),
        bert_layer,
        tf.keras.layers.Lambda(lambda x: x[:, 0, :]),
        tf.keras.layers.Dense(constants.VOCAB_SIZE, activation=tf.nn.softmax)
])

model.build(input_shape=(None, constants.MAX_SEQ_LENGTH - 1))

model.compile(loss='categorical_crossentropy', optimizer=tf.optimizers.Adam(lr=0.00001), metrics=['accuracy'])

print(model.summary())

history = model.fit(
            training_set,
            training_label,
            epochs=epochs,
            validation_data=(testing_set, testing_label),
            verbose=1,
            callbacks=[cp_callback]
        )

Can you help me with this problem? Thank you