search

VIPL-SLP / VAC_CSLR

Visual Alignment Constraint for Continuous Sign Language Recognition. ( ICCV 2021)
https://openaccess.thecvf.com/content/ICCV2021/html/Min_Visual_Alignment_Constraint_for_Continuous_Sign_Language_Recognition_ICCV_2021_paper.html
Apache License 2.0
116 stars 19 forks source link

Final accuracy #1

Closed hulianyuyy closed 2 years ago

hulianyuyy commented 2 years ago

I want to make sure that you report 22.1 Dev WER and 23.0 Test WER, while 21.2 Dev WER and 22.3 Test WER of released pretrained model ? Thanks in advance for response!

hulianyuyy commented 2 years ago

I have witnessed the latest version of your paper where results are updated. This issue is to be closed.

hulianyuyy commented 2 years ago

However, i'd like to ask what it is to make you boost it from 22.1 to 21.2 compared to the last version of code? Thanks in advance.

ycmin95 commented 2 years ago

@hulianyuyy We add ablations about the temporature and loss weight in supplementary, which can be found in Table 7 & 8 of our arXiv version.

hulianyuyy commented 2 years ago

May i also ask that the evaluation tool (python or scilte) will or won't affect the performance?

ycmin95 commented 2 years ago

@hulianyuyy From my experience, the evaluation differences between these tools about WER are less than 0.1%.

hulianyuyy commented 2 years ago

Then how could i reproduce the results of iterative training as mentioned in tab.1 in the paper ?

ycmin95 commented 2 years ago

@hulianyuyy You can follow their papers details and try to implement them, we didn't revised this part of codes. Besides, there still a performance gap between my implementation and their results and we didn't pay much attention on this.

hulianyuyy commented 2 years ago

Many thanks for your reply of your work which will benefit the community a lot.