Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
MIT License
661
stars
90
forks
source link
Remove one character entities on slang dictionary #22
The slang dictionary automatically removes one-letter long words such as 'n->and' and 'u->you'. I think this should be optional.