search

twairball / t2t_wmt_zhen

NMT for chinese-english using tensor2tensor
MIT License
47 stars 12 forks source link

BLEU evaluation on Chinese #4

Closed ChiZhangRIT closed 6 years ago

ChiZhangRIT commented 6 years ago

Thanks for sharing the code.

I was trying to run the code, but this is what I got:

./main/gen_orig.sh: line 6: t2t-datagen: command not found

BTW, how did you run BLEU evaluation on Chinese sentences? Did you use any tokenizer to get BLEU score?

twairball commented 6 years ago

hi, this code was based on a previous version of tensor2tensor -- it was merged into main repo so you can run the translation experiment there.

Default tensor2tensor tokenizer was used, based on subword encoding (sennrich et al)