search

google-research / pegasus

Apache License 2.0
1.61k stars 316 forks source link

Length controlled summarization #92

Closed sadiayousafzai036 closed 4 years ago

sadiayousafzai036 commented 4 years ago

Hi,

What changes can be made to implement length based summarization?

JingqingZ commented 4 years ago

Hi, could you elaborate what is length-controlled summarization in your context?

sadiayousafzai036 commented 4 years ago

For example, if you specify length like 20 words, the pegasus model should output an abstractive summary of 20 words only of the input text.

JingqingZ commented 4 years ago

You can specify the maximum length of output summaries by max_output_len (the summaries can be incomplete if this is too small) or encourage longer/shorter summaries by beam_alpha in public_params.py. But I doubt if you can control the output length as precisely as you would like, for example, a complete summary with 20 words in particular.

sadiayousafzai036 commented 4 years ago

Got it. My question was how the model could be modified? what if i plan to migrate the model from a specific layer using transfer based learning and implement length constraint output summarized text using seq2seq approach?

sadiayousafzai036 commented 4 years ago

Also, do you have an image or picture of detailed architecture of pegasus model? That would be really helpful.

JingqingZ commented 4 years ago

Hi, max_output_len and beam_alpha can be modified in the file just mentioned above. The code of the model is available https://github.com/google-research/pegasus/tree/master/pegasus/models.

The diagram of the architecture is available in our paper. Please refer to the paper link in README.

ShirleyHan6 commented 4 years ago

Hi @JingqingZ , may I check with you if the gold summary is also truncated to max_output_len for computing ROUGE?

JingqingZ commented 4 years ago

Hi @JingqingZ , may I check with you if the gold summary is also truncated to max_output_len for computing ROUGE?

yes

sadiayousafzai036 commented 4 years ago

Thankyou @JingqingZ for your response. How the beam_alpha parameter affects length of summarized text? increasing the value of beam_alpha will result in larger summaries?

JingqingZ commented 4 years ago

Thankyou @JingqingZ for your response. How the beam_alpha parameter affects length of summarized text? increasing the value of beam_alpha will result in larger summaries?

@sadiayousafzai036 As far as I remember, yes, larger beam_alpha longer output, but beam_alpha should be between 0 and 1 and typically it is around 0.6-0.9. See section 7 and equation 14 in https://arxiv.org/pdf/1609.08144.pdf