search

microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
https://aka.ms/GeneralAI
MIT License
19.76k stars 2.52k forks source link

MiniLM - Question Generation, Extractive Summarization #180

Closed anshoomehra closed 4 years ago

anshoomehra commented 4 years ago

Describe Model I am using (UniLM, MiniLM, LayoutLM ...):

Team,

  1. Has the pre-trained MiniLM for Question Generation been Released? If not, can you please share and if feasible, also share the delta results in performance between MiniLM and UniLM?

  2. I was curious if you have explored Extractive Summarization using MiniLM (or even UniLM) and any thoughts, pre-trained models on this front?

Great work as always, much appreciated your time and attention.

Best, Anshoo

donglixp commented 4 years ago

For the second question, using extractive summarization or abstractive summarization depends on the dataset itself. The model also works well for extractive summarization, if the task tends to directly copy the input sentences. The training objective is to predict whether a sentence should be included in the output summary. One application example is shown in Figure 2 of https://arxiv.org/pdf/1911.03324.pdf .