search

ictnlp / OR-NMT

Source Code for ACL2019 paper <Bridging the Gap between Training and Inference for Neural Machine Translation>
41 stars 10 forks source link

Scheduled sampling in Sentence oracle #5

Closed haorannlp closed 4 years ago

haorannlp commented 4 years ago

Hi Zhang Wen,

Did you use scheduled sampling when training with sentence oracle? In your code, you seem to use the sentence oracle for teacher forcing instead of sampling from either the gold target with probability p or the sentence oracle with probability (1-p). Thanks.

zhang-wen commented 4 years ago

@haorannlp hi, no, we do not use the sentence oracle for the teacher forcing. We just modify the input of the decoder. For the sentence oracle. But yes, as you said, the gold target with probability p or the sentence oracle with probability (1-p) is used as the input of the decoder to predict the ground-truth sentence. The code now makes scheduled sampling at the sentence level

Line 344-346 in: https://github.com/ictnlp/OR-NMT/blob/2effd15a77595ebfcf40bc841c66b3d873784ff5/OR-Transformer/fairseq/models/oracle_transformer.py#L344

which may have some problems, but the impact will be small I think. We will update the code to make scheduled sampling at the word level. Thank you for your reminding.

haorannlp commented 4 years ago

@zhang-wen I see, thanks.