Open brunopistone opened 3 years ago
I'm getting the following error when I'm trying to train a model for generating the next word in a sentence:
Paddings must be non-negative for 'Pad' (op: 'Pad') with input shapes: [1071,768], [2,2] and with computed input tensors: input[1] = <[0 -559][0 0]>.
The pre-trained model used is multi_cased_L-12_H-768_A-12.
This is the code:
bert_layer = bert.BertModelLayer.from_params(self.bert_params, name="bert") bert_params = bert.params_from_pretrained_ckpt(model_path) tokenizer = bert.bert_tokenization.FullTokenizer(os.path.join(model_path, "vocab.txt"), do_lower) model = tf.keras.Sequential([ tf.keras.layers.Input(shape=(constants.MAX_SEQ_LENGTH - 1,), dtype='int32', name='input_ids'), bert_layer, tf.keras.layers.Lambda(lambda x: x[:, 0, :]), tf.keras.layers.Dense(constants.VOCAB_SIZE, activation=tf.nn.softmax) ]) model.build(input_shape=(None, constants.MAX_SEQ_LENGTH - 1)) model.compile(loss='categorical_crossentropy', optimizer=tf.optimizers.Adam(lr=0.00001), metrics=['accuracy']) print(model.summary()) history = model.fit( training_set, training_label, epochs=epochs, validation_data=(testing_set, testing_label), verbose=1, callbacks=[cp_callback] )
Can you help me with this problem? Thank you
I'm getting the following error when I'm trying to train a model for generating the next word in a sentence:
The pre-trained model used is multi_cased_L-12_H-768_A-12.
This is the code:
Can you help me with this problem? Thank you