search

ruotianluo / self-critical.pytorch

Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.
MIT License
991 stars 278 forks source link

transformer性能 #252

Closed hasky123 closed 3 years ago

hasky123 commented 3 years ago

您好，我使用 transformer.yml训练只能达到 Bleu_1: 0.749 Bleu_2: 0.584 Bleu_3: 0.443 Bleu_4: 0.336 METEOR: 0.271 ROUGE_L: 0.553 CIDEr: 1.092 远远不及您提及的分数，请问您训练参数是如何设置的？

ruotianluo commented 3 years ago

这是beam search的结果？

Ruotian Luo

On May 23, 2021, at 3:37 AM, xiantwo @.***> wrote:

 您好，我使用 transformer.yml训练只能达到 Bleu_1: 0.749 Bleu_2: 0.584 Bleu_3: 0.443 Bleu_4: 0.336 METEOR: 0.271 ROUGE_L: 0.553 CIDEr: 1.092 远远不及您提及的分数，请问您训练参数是如何设置的？

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

hasky123 commented 3 years ago

@ruotianluo 是的，我又跑了一次，结果如下 Bleu_1: 0.757 Bleu_2: 0.591 Bleu_3: 0.450 Bleu_4: 0.340 METEOR: 0.273 ROUGE_L: 0.557 CIDEr: 1.103 beam size = 2

ruotianluo commented 3 years ago

没有beam saerch是多少？另外试试beam size 5>

hasky123 commented 3 years ago

@ruotianluo 这是没有beam search的结果 Bleu_1: 0.748 Bleu_2: 0.582 Bleu_3: 0.437 Bleu_4: 0.325 METEOR: 0.271 ROUGE_L: 0.553 CIDEr: 1.078 这是beam size =5 的结果 Bleu_1: 0.751 Bleu_2: 0.588 Bleu_3: 0.450 Bleu_4: 0.345 METEOR: 0.273 ROUGE_L: 0.556 CIDEr: 1.092

ruotianluo commented 3 years ago

确实比我这要低。你用的什么机器？

ruotianluo commented 3 years ago

我这没有beam search是CIDEr 1.09+(validation)

hasky123 commented 3 years ago

@ruotianluo tesla p100 pytorch1.5 + python3.7。另外，我也在 pytorch1.4和1.8跑过，结果都差不多。 分数远远不如一些论文提到的性能，我很好奇这是什么原因？

ruotianluo commented 3 years ago

我这个肯定是能跑到的，我最近刚跑过。如果config是一样的照理说不会差这么多。你这个evaluate的是best ckpt还是last ckpt。

hasky123 commented 3 years ago

histories文件显示我最好验证集性能是 epoch13 iteration150000 cider1.0816737178817097 best.ckpt

ruotianluo commented 3 years ago

我也不知道问题出在哪了。