When pointer_gen=True, why did you change the decoder inputs using encoder oovs?

abisee / pointer-generator

Code for the ACL 2017 paper "Get To The Point: Summarization with Pointer-Generator Networks"

Other

2.18k stars 812 forks source link

When pointer_gen=True, why did you change the decoder inputs using encoder oovs? #101

Open lan2720 opened 6 years ago

lan2720 commented 6 years ago

Hi, I have a question about the data preprocessing. I mean this line: https://github.com/abisee/pointer-generator/blob/master/batcher.py#L71 Here you create enc_input_extend_vocab and modify target, but dec_input unchanged. Why didn't you replace the UNK in dec_input with oovs? Is there any consideration?

bhomass commented 5 years ago

I am looking at the same issue. I believe it is because the word embedding is limited to the vocab size. These embeddings are updated during training, therefore the words in the article (fed into the encoder) must be derived from the embedding matrix. Whereas, the target words are not in the path of back propagation, and can point to the extended words, which are simply indices, with no vector representation.