Are there any differences when using "predict_raw_text" or not in DrQA model?

stanfordnlp / coqa-baselines

The baselines used in the CoQA paper

MIT License

176 stars 52 forks source link

Are there any differences when using "predict_raw_text" or not in DrQA model? #13

Closed Milozms closed 5 years ago

Milozms commented 5 years ago

I saw in README that reading comprehension model sets predict_raw_text=True, while the pipeline model sets predict_raw_text=False. Are there any differences?

danqi commented 5 years ago

When predict_raw_text is set to True, it will map the span prediction to the original text (using offsets) and extract the corresponding text for final evaluation (instead of concatenating all words using whitespace). This is a common practice for both SQuAD and CoQA and it usually works slightly better.

danqi commented 5 years ago

Oh, for the pipeline model, we need that predicted rationale is tokenized so it can be fed into the seq2seq model directly so we set predict_raw_text=False.