search

VIPL-SLP / VAC_CSLR

Visual Alignment Constraint for Continuous Sign Language Recognition. ( ICCV 2021)
https://openaccess.thecvf.com/content/ICCV2021/html/Min_Visual_Alignment_Constraint_for_Continuous_Sign_Language_Recognition_ICCV_2021_paper.html
Apache License 2.0
116 stars 19 forks source link

Visualizing the predicted alignments #33

Open snalyami opened 1 year ago

snalyami commented 1 year ago

Thank you so much for releasing your code!

I'm trying to visualize the predicted alignments but the timings in the out.output-hypothesis-dev.ctm.sgml and the out.output-hypothesis-test.ctm.sgml files don’t make sense. The timings t1+t2 indicate the start time + end time? This means that the gloss es overlap. Also I get the same timings for all the samples e.g. 0.000+0.010 0.010+0.030 0.020+0.050 0.030+0.07

Example from the out.output-hypothesis-test.ctm.sgml file:

<SYSTEM title="./work_dir/baseline_res18_SD_VAC_Phoenix/out.output-hypothesis-test.ctm" ref_fname="./work_dir/baseline_res18_SD_VAC_Phoenix/tmp.stm" hyp_fname="./work_dir/baseline_res18_SD_VAC_Phoenix/out.output-hypothesis-test.ctm" creation_date="Sun Dec 11 22:13:30 2022" format="2.4" frag_corr="FALSE" opt_del="FALSE" weight_ali="FALSE" weight_filename="">
<SPEAKER id="signer04">
<PATH id="(signer04-000)" word_cnt="6" file="01april_2010_thursday_heute_default-5" channel="1" sequence="0" R_T1="0.000" R_T2="inf" word_aux="h_t1+t2">
C,"aber","aber",0.000+0.010:S,"freuen","woche",0.010+0.030:C,"morgen","morgen",0.020+0.050:C,"sonne","sonne",0.030+0.070:S,"selten","kaum",0.040+0.090:C,"regen","regen",0.050+0.110
</PATH>
<PATH id="(signer04-001)" word_cnt="7" file="01april_2010_thursday_tagesschau_default-7" channel="1" sequence="1" R_T1="0.000" R_T2="inf" word_aux="h_t1+t2">
C,"samstag","samstag",0.000+0.010:C,"wechselhaft","wechselhaft",0.010+0.030:C,"besonders","besonders",0.020+0.050:C,"freundlich","freundlich",0.030+0.070:D,"nordost",,:S,"bisschen","nord",0.040+0.090:S,"bereich","ix",0.060+0.130
</PATH>
<PATH id="(signer04-002)" word_cnt="7" file="01april_2010_thursday_tagesschau_default-8" channel="1" sequence="2" R_T1="0.000" R_T2="inf" word_aux="h_t1+t2">
C,"sonntag","sonntag",0.000+0.010:C,"regen","regen",0.010+0.030:C,"teil","teil",0.020+0.050:C,"gewitter","gewitter",0.030+0.070:C,"suedost","suedost",0.040+0.090:D,"durch",,:C,"regen","regen",0.050+0.110
</PATH>
ycmin95 commented 1 year ago

Hi, I'd like to clarify that we do not currently utilize SGML files to visualize the alignment. However, you may refer to the discussion about the alignment in this issue. If you have any further inquiries or difficulties, please do not hesitate to post them here - we'd be happy to help!