Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
Right now unpack_contractions check for ' e.g. you're. But sometime in text, ’ is used instead of ' e.g. you’re. This PR should would allow checking for both cases.
cbaziotis
commented
1 year ago
Right now unpack_contractions check for ' e.g. you're. But sometime in text, ’ is used instead of ' e.g. you’re. This PR should would allow checking for both cases.