Open samjmolyneux opened 1 month ago
Try atleast just adding a different method for tokenization to see how well it works, how fast it is, and what the difference in the number of total tokens ends up being once you get to the tfidf vectorizer.
Try atleast just adding a different method for tokenization to see how well it works, how fast it is, and what the difference in the number of total tokens ends up being once you get to the tfidf vectorizer.