Errors during evaluation

[yokopsycho]

Thank you very much for the excellent program you have created.

I have trained 5 epochs in exactly the same way, but all the bots return "Good morning".

It seems to be a simple mistake, but I would appreciate any feedback.

In addition, I have modified the following part of the program so that it will work on 7/29/2021.

1.config.py

model_name = "bert-base-japanese-whole-word-masking"

model_name = "cl-tohoku/bert-base-japanese-whole-word-masking" 2.tokenizer.py

from transformers.tokenization_bert_japanese import BertJapaneseTokenizer

from transformers import BertJapaneseTokenizer

Sincerely yours.

The following is what I did during the training.

python main.py INFO:root: Initializing INFO:root:Preparing training data INFO:root:Define Models Some weights of the model checkpoint at cl-tohoku/bert-base-japanese-whole-word-masking were not used when initializing BertEncoder: ['cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.seq_relationship.bias']

• This IS expected if you are initializing BertEncoder from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing BertEncoder from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). INFO:root:Define Loss and Optimizer INFO:root:Start Training Epoch: 1: 100%|██████████| 54882/54882 [3:42:46<00:00, 4.11it/s, Loss: 2.24628] Saved Model おはようございます。 Epoch: 2: 100%|██████████| 54882/54882 [3:41:45<00:00, 4.12it/s, Loss: 2.35587] Saved Model おはようございます。 Epoch: 3: 100%|██████████| 54882/54882 [3:41:52<00:00, 4.12it/s, Loss: 2.28201] Saved Model おはようございます Epoch: 4: 100%|██████████| 54882/54882 [3:43:05<00:00, 4.10it/s, Loss: 2.19522] Saved Model おはようございます Epoch: 5: 100%|██████████| 54882/54882 [3:41:27<00:00, 4.13it/s, Loss: 2.32014] Saved Model おはようございます

The following is errors of my chat bots ckpt_4 (epoch 5)   You>こんにちは。 BOT>おはようございます You>暑いですね。 BOT>おはようございます You>元気ですか？ BOT>おはようございます You>今日雨フルらしいよ BOT>おはようございます

ckpt (original model)

You>おはよう BOT>おはようございます You>元気ですか？ BOT>おはようございます You>むむ BOT>おはようございます You>

[reppy4620]

I apologize for the extremely delayed response. I was not aware of the issue that had arisen. This problem may be alleviated by modifying the loss in the loss.py file to incorporate LabelSmoothing. Additionally, it may be necessary to thin out the frequently occurring responses within the dataset.