search

graykode / xlnet-Pytorch

Simple XLNet implementation with Pytorch Wrapper
https://arxiv.org/pdf/1906.08237.pdf
Apache License 2.0
577 stars 107 forks source link

position-wise feedforward only one linear layer? #16

Open yuanenming opened 3 years ago

yuanenming commented 3 years ago

https://github.com/graykode/xlnet-Pytorch/blob/cb793a1c75bdc59e3360f04ec641af726719811f/xlnet.py#L163

In your implementation, the FFN module only has one linear layer. is it a bug?