Which tokenizer does this model use?

bloomsburyai / question-generation

Neural text-to-text question generation

MIT License

217 stars 52 forks source link

Closed zzj0402 closed 4 years ago

zzj0402 commented 4 years ago

As title.

tomhosking commented 4 years ago

Tokenisation happens here - it uses the NLTK tokenizer.