artitw / text2text

Text2Text: Crosslingual NLP/G toolkit
https://discord.gg/eHaaUuWpTc
Other
283 stars 33 forks source link

How does the summarization work? #4

Closed Hellisotherpeople closed 4 years ago

Hellisotherpeople commented 4 years ago

This looks like it's doing abstractive summarization, but occasionally it can do pure extractive summarization.

Can you confirm - and explain the exact methodology used in the summarization module?

artitw commented 4 years ago

It’s abstractive and based on the pre-training transformers like BERT. For more details, see https://arxiv.org/abs/1905.03197

For an introduction to transformers, see https://youtu.be/iRT6C7R7NHk