Replace custom tokenization with cadmium native one. - Githubissues

cadmiumcr / language_detector

Detects the language of a text sample

MIT License

7 stars 0 forks source link

Replace custom tokenization with cadmium native one. #2

Open rmarronnier opened 5 years ago

rmarronnier commented 5 years ago

Investigate how the RegexTokenizer could be used along Cadmium::n-gram