Open divilian opened 2 years ago
Turns out that we are, in two different places, doing versions of the same thing:
if we have a CountVectorizer, call:
vectorizer.fit_transform(all_threads).toarray()
if we have a Tokenizer, on which we have called .git_on_texts(), call:
tokenizer.texts_to_matrix(threads, mode=METHOD)
Only the first of these actually cares about useBigrams. (!!)
useBigrams
Turns out that we are, in two different places, doing versions of the same thing:
if we have a CountVectorizer, call:
if we have a Tokenizer, on which we have called .git_on_texts(), call:
Only the first of these actually cares about
useBigrams. (!!)