search

ghpaetzold / questplusplus

Pipelined quality estimation.
49 stars 14 forks source link

Is there any special requirement that we need to add when running chinese data? #49

Open Shireen35 opened 3 years ago

Shireen35 commented 3 years ago

When i use questplusplus for english-spanish/spansih-english it works fine. But chinese-english/english-chinese does not !!! .eng running tokenizer running tokenizer on /home/shireen/questplusplus/based/baseline/input/chinese/source.zh ./scripts/truecase.perl|./lang_resources/english/truecase-model.en ./scripts/tokenizer.perl running truecase done tokenizing... perl ./scripts/tokenizer.perl -a -l en /home/shireen/questplusplus/based/baseline/input/chinese/source.zh.tok.temp done /home/shireen/questplusplus/based/baseline/input/chinese/source.zh.tok running tokenizer running tokenizer on /home/shireen/questplusplus/based/baseline/input/english/target.eng null|null null As u can see the tokenizer runs null|null instead of the script command .i don't know why??? Pls help!!! WHen i run the tokenizer individually it runs fine!!