where is the ../data/resources/chinese_stop_words.txt

[520jefferson]

how can i find this file? @chujiezheng

[chujiezheng]

You can use any stop word list if appropriate, such as this

[520jefferson]

You can use any stop word list if appropriate, such as this Thanks, i already use this: https://blog.csdn.net/shijiebei2009/article/details/39696571

[520jefferson]

@chujiezheng is the logs normal ? when i download pytorch version of -- BERT-wwm, Chinese after i start the program i find this below

01/28/2021 10:02:18 - INFO - main - ### missing keys:['embed.weight', 'classifier.weight', 'classifier.bias', 'reshape.weight', 'reshape_know.weight', 'reshape_know.bias'] 01/28/2021 10:02:18 - INFO - main - ### unexpected keys:['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias'] 01/28/2021 10:02:18 - INFO - main - ### error msgs:[] 01/28/2021 10:02:24 - WARNING - pytorch_pretrained_bert.optimization - t_total value of -1 results in schedule not being applied 01/28/2021 10:02:24 - INFO - main - Running training 01/28/2021 10:02:24 - INFO - main - Num post-response pairs = 27550 01/28/2021 10:02:24 - INFO - main - Batch size = 2 01/28/2021 10:02:24 - INFO - main - Num steps = 13775 Epoch: 0%| | 0/8 [00:00<?, ?it/s]

[chujiezheng]

If the training works, I think you can ignore it. These messages tell us that the parameter items saved the checkpoint do not exactly match those defined in the model.

[520jefferson]

If the training works, I think you can ignore it. These messages tell us that the parameter items saved the checkpoint do not exactly match those defined in the model.

Thanks for your reply. But when the training begin the Losses look erratic. Is this normal?

01/28/2021 10:05:21 - INFO - main - loss@999:0.03600410372018814 01/28/2021 10:08:12 - INFO - main - loss@1999:0.017265524715185165 01/28/2021 10:11:04 - INFO - main - loss@2999:0.02728239819407463 01/28/2021 10:13:55 - INFO - main - loss@3999:0.04541652649641037 01/28/2021 10:16:48 - INFO - main - loss@4999:0.028780410066246986 01/28/2021 10:19:39 - INFO - main - loss@5999:0.026078417897224426 01/28/2021 10:22:31 - INFO - main - loss@6999:0.018436497077345848 01/28/2021 10:25:22 - INFO - main - loss@7999:0.028281718492507935 01/28/2021 10:28:25 - INFO - main - loss@8999:0.009581252001225948 01/28/2021 10:31:16 - INFO - main - loss@9999:0.02277129888534546 01/28/2021 10:34:07 - INFO - main - loss@10999:0.016799500212073326 01/28/2021 10:36:59 - INFO - main - loss@11999:0.04521077871322632 01/28/2021 10:39:52 - INFO - main - loss@12999:0.028096113353967667 Epoch: 12%|█▎ | 1/8 [39:41<4:37:51, 2381.58s/it]01/28/2021 10:44:58 - INFO - main - loss@999:0.017184492200613022 01/28/2021 10:47:49 - INFO - main - loss@1999:0.03952294960618019 01/28/2021 10:50:41 - INFO - main - loss@2999:0.01909543387591839 (py3.6-wlj) [work@zjk3-chatbot-g8v100-node02 membertret]$ tail -f logs.film.train.0127 01/28/2021 10:22:31 - INFO - main - loss@6999:0.018436497077345848 01/28/2021 10:25:22 - INFO - main - loss@7999:0.028281718492507935 01/28/2021 10:28:25 - INFO - main - loss@8999:0.009581252001225948 01/28/2021 10:31:16 - INFO - main - loss@9999:0.02277129888534546 01/28/2021 10:34:07 - INFO - main - loss@10999:0.016799500212073326 01/28/2021 10:36:59 - INFO - main - loss@11999:0.04521077871322632 01/28/2021 10:39:52 - INFO - main - loss@12999:0.028096113353967667 Epoch: 12%|█▎ | 1/8 [39:41<4:37:51, 2381.58s/it]01/28/2021 10:44:58 - INFO - main - loss@999:0.017184492200613022 01/28/2021 10:47:49 - INFO - main - loss@1999:0.03952294960618019 01/28/2021 10:50:41 - INFO - main - loss@2999:0.01909543387591839 01/28/2021 10:53:32 - INFO - main - loss@3999:0.012951666489243507 01/28/2021 10:56:23 - INFO - main - loss@4999:0.03446248173713684

[chujiezheng]

Sorry, I am not sure.

If you find that the model is trained badly, you can check the following potential problems:

• The requirements, such as pytorch-pretrained-bert
• The checkpoint, which I remembered was download from this repo
• If the above is all right, you had better check carefully the process of loading parameters. Maybe some parameters such as word embeddings are not loaded rightly due to the unmatched name.

[520jefferson]

Sorry, I am not sure.

If you find that the model is trained badly, you can check the following potential problems:

• The requirements, such as pytorch-pretrained-bert
• The checkpoint, which I remembered was download from this repo
• If the above is all right, you had better check carefully the process of loading parameters. Maybe some parameters such as word embeddings are not loaded rightly due to the unmatched name.

yes，i use this

it sames better: Epoch: 62%|██████▎ | 5/8 [3:17:16<1:58:16, 2365.65s/it]01/28/2021 13:22:32 - INFO - main - loss@999:0.007644557394087315 01/28/2021 13:25:23 - INFO - main - loss@1999:0.007134606130421162 01/28/2021 13:28:14 - INFO - main - loss@2999:0.0028984006494283676 01/28/2021 13:31:06 - INFO - main - loss@3999:0.0042317588813602924 01/28/2021 13:33:58 - INFO - main - loss@4999:0.0011935405200347304 01/28/2021 13:36:49 - INFO - main - loss@5999:0.014066444709897041 01/28/2021 13:39:41 - INFO - main - loss@6999:0.010450128465890884 01/28/2021 13:42:33 - INFO - main - loss@7999:0.023439526557922363 01/28/2021 13:45:24 - INFO - main - loss@8999:0.00323661626316607 01/28/2021 13:48:16 - INFO - main - loss@9999:0.010449349880218506 01/28/2021 13:51:07 - INFO - main - loss@10999:0.0009684869437478483 01/28/2021 13:53:59 - INFO - main - loss@11999:0.000689977256115526 01/28/2021 13:56:51 - INFO - main - loss@12999:0.027426261454820633 Epoch: 75%|███████▌ | 6/8 [3:56:40<1:18:50, 2365.33s/it]01/28/2021 14:01:57 - INFO - main - loss@999:0.0006691914750263095 01/28/2021 14:04:49 - INFO - main - loss@1999:0.007243728265166283 01/28/2021 14:07:40 - INFO - main - loss@2999:0.000343370164046064 01/28/2021 14:10:31 - INFO - main - loss@3999:0.024247396737337112 01/28/2021 14:13:23 - INFO - main - loss@4999:0.0001573079644003883 01/28/2021 14:16:14 - INFO - main - loss@5999:0.00498569943010807 01/28/2021 14:19:06 - INFO - main - loss@6999:0.00010840449249371886 01/28/2021 14:21:58 - INFO - main - loss@7999:0.0015040287980809808 01/28/2021 14:24:49 - INFO - main - loss@8999:0.014488743618130684 01/28/2021 14:27:40 - INFO - main - loss@9999:0.002513666870072484 01/28/2021 14:30:31 - INFO - main - loss@10999:0.0005113831721246243 01/28/2021 14:33:23 - INFO - main - loss@11999:0.0005174471880309284 01/28/2021 14:36:15 - INFO - main - loss@12999:0.002770752180367708 Epoch: 88%|████████▊ | 7/8 [4:36:04<39:24, 2364.91s/it]

[520jefferson]

@chujiezheng I wonder if i have the 10 million query - response pairs. i use the bert + know retrieval method, and k-v is got from the entity link, this just check the entity and have a normal description. Then the result will be better ? because many of the query-response doesn't have knowledge form like the experiment corpus.

Do you have any suggestion if i want to use knowledge generation or retrieve

[520jefferson]

$ sh test_film.sh KdConv/benchmark/membertret 02/05/2021 11:12:21 - INFO - main - Running testing 02/05/2021 11:12:21 - INFO - main - Num post-response pairs = 3860 02/05/2021 11:12:21 - INFO - main - Batch size = 16 02/05/2021 11:12:21 - INFO - main - Start evaluating test set restart, 241 batches and 4 left Building prefix dict from the default dictionary ... 02/05/2021 11:12:22 - DEBUG - jieba - Building prefix dict from the default dictionary ... Loading model from cache /tmp/jieba.cache 02/05/2021 11:12:22 - DEBUG - jieba - Loading model from cache /tmp/jieba.cache Loading model cost 0.692 seconds. 02/05/2021 11:12:23 - DEBUG - jieba - Loading model cost 0.692 seconds. Prefix dict has been built successfully. 02/05/2021 11:12:23 - DEBUG - jieba - Prefix dict has been built successfully. Test Result: bleu-1: 0.848291 bleu-2: 0.815466 bleu-3: 0.797040 bleu-4: 0.784814 distict_1: 0.087366 distict_2: 0.323164 distict_3: 0.534185 distict_4: 0.662428 hits@1: 0.712176 hits@3: 0.939119

I get better result after i run the membertret , it's normal ?

[chujiezheng]

I think it is normal. We did not do too much parameter-tuning when doing experiments.

[520jefferson]

I think it is normal. We did not do too much parameter-tuning when doing experiments. 生成模型得到的结果如下： 看生成的结果132分钟都搞成107分钟了。是不是语言模型影响太大了？正确知识并没有融入进去

bleu-1: 0.270373 bleu-2: 0.140048 bleu-3: 0.082904 bleu-4: 0.050558 distict_1: 0.029305 distict_2: 0.083257 distict_3: 0.158718 distict_4: 0.243424 hits@1: 0.201554 hits@3: 0.423834 perplexity: 25.446977

resp: 是 的 ， 它 是 2001 年 12 月 28 日 上映 的 ， 很 感人 的 电影 。 gen: 看过 啊 ， 它 是 一部 很 好看 的 电影 。

resp: 是 的 ， 是 一部 很 好看 的 家庭 ， 亲情 剧 ， 我 记得 片长 也 很长 的 。 gen: 是 的 ， 它 是 由 东宝 映画 制作 的 电影 。

resp: 是 的 132 分钟 ， 你 就 准备 好 哭 吧 。 gen: 片长 是 107 分钟 ， 你 知道 这部 电影 的 主演 都 有 谁 吗 ？

resp: 电影 的 票房 怎么样 啊 ？ gen: 是 的 ， 这部 电影 是 在 哪里 制片 的 呢 ？

resp: 这 不 清楚 ， 但 我 知道 它 获得 了 2002 年 青年 艺术家 奖 等 诸多 奖项 ， 以及 2002 年 奥斯卡 美国 >电影 学院奖 等 多项 奖项 提名 。 gen: 这个 我 不 清楚 。

[chujiezheng]

RNN的语言模型效果本身就不太好。可以用GPT或者加copy试试

[520jefferson]

RNN的语言模型效果本身就不太好。可以用GPT或者加copy试试

后续实验下，另外请教下生成模型的指标hits@1或hits@3，我看是根据loss算的？不知道原理是啥？

[chujiezheng]

选择候选回复中loss最低（也就是ppl最低）的那一句