input=/home/shireen/questplusplus/baseline/input output=/home/shireen/questplusplus/baseline/output
folder created /home/shireen/questplusplus/baseline/input/chinese
folder created /home/shireen/questplusplus/baseline/input/english
source input:input/source.zh
target input:input/target.eng
copying input to /home/shireen/questplusplus/baseline/input/chinese/source.zh
copying input to /home/shireen/questplusplus/baseline/input/english/target.eng
running tokenizer
running tokenizer on /home/shireen/questplusplus/baseline/input/chinese/source.zh
./scripts/truecase.perl|./lang_resources/english/truecase-model.en
./scripts/tokenizer.perl
running truecase
done
tokenizing...
perl ./scripts/tokenizer.perl -a -l en /home/shireen/questplusplus/baseline/input/chinese/source.zh.tok.temp
done
/home/shireen/questplusplus/baseline/input/chinese/source.zh.tok
running tokenizer
running tokenizer on /home/shireen/questplusplus/baseline/input/english/target.eng
null|null
null
running truecase
java.io.IOException: Stream closed
The second time tokeniser function of is not visble its null please help me as I don't understand why is this the case?For chinese it works not for english????
input=/home/shireen/questplusplus/baseline/input output=/home/shireen/questplusplus/baseline/output folder created /home/shireen/questplusplus/baseline/input/chinese folder created /home/shireen/questplusplus/baseline/input/english source input:input/source.zh target input:input/target.eng copying input to /home/shireen/questplusplus/baseline/input/chinese/source.zh copying input to /home/shireen/questplusplus/baseline/input/english/target.eng running tokenizer running tokenizer on /home/shireen/questplusplus/baseline/input/chinese/source.zh ./scripts/truecase.perl|./lang_resources/english/truecase-model.en ./scripts/tokenizer.perl running truecase done tokenizing... perl ./scripts/tokenizer.perl -a -l en /home/shireen/questplusplus/baseline/input/chinese/source.zh.tok.temp done /home/shireen/questplusplus/baseline/input/chinese/source.zh.tok running tokenizer running tokenizer on /home/shireen/questplusplus/baseline/input/english/target.eng null|null null running truecase java.io.IOException: Stream closed
The second time tokeniser function of is not visble its null please help me as I don't understand why is this the case?For chinese it works not for english????