Selecting answer span during preprocessing

[valsworthen]

Hello,

I have trouble understanding how exactly the preprocessing script selects the answer spans from the answer text. From what I understand, the function fix_span loops over all matches of the answer text and tries to select the one that better "sticks" to individual tokens. In the case where the answer perfectly matches some tokens, the first occurrence is returned.

Is that right?

Thanks!

[qipeng]

Yes, I think that's correct. Although one could argue whether the first match is always ideal, I agree.