Closed Lomadriel closed 9 years ago
Added a tokenizer, it supports several characters and it also has a limit.
We should also remove the split function which is now obsolete.
Added a tokenizer, it supports several characters and it also has a limit.
We should also remove the split function which is now obsolete.