search

atulkum / pointer_summarizer

pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"
Apache License 2.0
907 stars 242 forks source link

pointer generator model starts overfitting #28

Closed priyanks179 closed 5 years ago

priyanks179 commented 5 years ago

I am trying to train pointer generator network . After training for 10k iterations it starts overfitting. Any suggestions on why this might be happening. Note- 1==50 iterations adagrad 52k

atulkum commented 5 years ago

make sure you turn off the coverage loss initially.

priyanks179 commented 5 years ago

make sure you turn off the coverage loss initially.

bro i am not using coverage loss at all . And model that i am using is exactly same as yours . Just the thing is that i am fixing max_enc_steps to 200 and max_dec_steps to 30 .

priyanks179 commented 5 years ago

sir pls help coz i am trying to solve this problem for a month and i am not able to figure it out .

priyanks179 commented 5 years ago

Sir the link to repo is below:- https://github.com/priyanks179/pointer-gen-network

A-Rain commented 4 years ago

@atulkum Hello sir, I'm confuse why we should turn off coverage loss initially

atulkum commented 4 years ago

If you turn on coverage initially the training won't converge. Intuitively think about coverage as one more constraint on optimization which will restrict the space you can explore. That is the reason first we explore the loss space without the restriction and once it converge a bit we apply the coverage.