Which Tokenizer was usked to split raw sentences

cambridgeltl / MTL-Bioinformatics-2016

Creative Commons Attribution 4.0 International

217 stars 92 forks source link

Which Tokenizer was usked to split raw sentences #4

Open ibrahimishag opened 3 years ago

ibrahimishag commented 3 years ago

Dear @GamalC , This is to first and foremost thank you sincerly for preparing the dataset. I wonder which tokenizer did you use when splitting raw sentences to tokens.?

Looking forward to your answer.