VIPL-SLP / VAC_CSLR

Visual Alignment Constraint for Continuous Sign Language Recognition. ( ICCV 2021)
https://openaccess.thecvf.com/content/ICCV2021/html/Min_Visual_Alignment_Constraint_for_Continuous_Sign_Language_Recognition_ICCV_2021_paper.html
Apache License 2.0
116 stars 19 forks source link

关于baseline复现结果不一致的问题 #14

Closed miaomiao9miao closed 2 years ago

miaomiao9miao commented 2 years ago

您好,我有一些关于实验代码的一些问题。 在您的论文表3中,baseline在DEV上的结果是25.4,我在代码中尝试将loss中的ConvCTC和Dist去掉来实现它,但是得到了:仅在epoch=40时,WER=24.8%,最终结果与表3中的结果相差较多,出现这样的结果是否是因为我疏忽了某些应该去掉的部分?

log.txt config.txt

miaomiao9miao commented 2 years ago

在我修改的其他的实验中,epoch=40时往往比最终结果要高1%左右,并且我在数据预处理中将Temporal Scaling去掉了,所以baseline结果应该更高一些,而不是更低 期待您的回复

ycmin95 commented 2 years ago

HI, @miaomiao9miao As shown in our recent update, the original implementation performs worse when adopting multiple GPU due to nn.Datapallel. Therefore, we adopt syncBN in our later experiments, and we release some experimental results in this update, more results will be released in our future journal version.

As you adopting single GPU (also modifying the batch size), which is expected to perform better than our previous conference version. We also release an updated baseline with 40 epochs training, and you can adopt this setting for comparison.