search

parkmichelle / movie-chatbot

1 stars 0 forks source link

Use nltk to tokenize instead of split #3

Open parkmichelle opened 5 years ago

parkmichelle commented 5 years ago

Look for instances where splitting text with .split(" "), tokenize with nltk instead or figure out how to tokenize with more than whitespace

parkmichelle commented 5 years ago

ie need to split by punctuation too?

parkmichelle commented 5 years ago

May also need to use porterstemmer, I think this might be a creative extension but not sure