search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
134.73k stars 26.94k forks source link

Running XLNet on Squad #1253

Closed LeonCrashCode closed 4 years ago

LeonCrashCode commented 5 years ago

❓ Questions & Help

This is the padding problem. In GLUE codes in the examples, the padding for XLNet is on the left of the input, but in Squad codes, the padding is on right. I was wondering which one is correct.

Also, the inputs of convert_examples_to_features are different in Glue and Squad, where Squad uses most of default values like pad_token, sep_token, pad_token_segment_id and cis_token_segment_id, but Glue use the value of tokenizer. which one is correct?

Or the example codes are out-of-date? Thanks

tbright17 commented 5 years ago

Seems like the run_squad script is in bad shape now. It just doesn't work.

hohoCode commented 5 years ago

same question.. also running this script with XLNet on Squad, is ~10 F1 points below BERT-Large-WWM. The difference in preprocessing as pointed out above could be one of the reasons.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.