Architecture for word-level Penn Treebank dataset

kimiyoung / transformer-xl

Apache License 2.0

3.59k stars 762 forks source link

Architecture for word-level Penn Treebank dataset #16

Open AlexGrinch opened 5 years ago

AlexGrinch commented 5 years ago

Hello!

Can you, please, provide the bash script for training Transformer-XL on PTB dataset with PyTorch?

Thanks!

pclucas14 commented 5 years ago

+1 for this! Thanks

likicode commented 5 years ago

+1! It would be really helpful to have the script for PTB dataset! @kimiyoung

world2vec commented 5 years ago

Dear authors, Could you publish details of model super parameters for PTB word? Thanks

kimiyoung commented 5 years ago

We decided not to include our PTB code in this repo because we believe PTB, being super small, is mainly a regularization game and is somewhat misleading for the development of language modeling. That said, we will try to find time in the future to clean and release our code only for reproducing our results.

sh0416 commented 3 years ago

@kimiyoung Did you mean that the reported score in the paper could not be reproducible?