Implementation XLNet with Pytorch for learning XLNet simple code with paper

[graykode]

I finished Simple XLNet implementation with Pytorch Wrapper You can see How XLNet Architecture work in pre-training with small batch size(=1) example. Also I added comment in code, so you can learn XLNet Architecture more easily Thanks for kimiyoung and zihangdai https://github.com/graykode/xlnet-Pytorch

[thomwolf]

This is a nice work but beware, BertTokenizer is not the same as the sentencepiece tokenizer of XLNet, you have to use the vocabulary of XLNet with Google SentencePiece library (haven't check the code more but I was intrigued by the dependency on pytorch-pretraine-bert).

[graykode]

@thomwolf Thanks for advice, first I'll edit my readme then I'll try to change the code to use the sensepice tomorrow. But right now, even if I use the Bert tokenizer, I think I'll be fine as a subword tokenizer. Thank you for always sharing a good project.