search

microsoft / ProphetNet

A research project for natural language generation, containing the official implementations by MSRA NLC team.
MIT License
654 stars 105 forks source link

fixed error in tokenization code for other datasets in readme #10

Closed ManavR123 closed 4 years ago

ManavR123 commented 4 years ago

In this code sample in the README which helps users tokenize their own data, there is a variable used new, which is not defined in the scope of the function. I believe the intended variable to be used is word_pieces.

yuyan2do commented 4 years ago

Thanks for pointing this typo.