susanli2016 / Machine-Learning-with-Python

Python code for common Machine Learning Algorithms
4.31k stars 4.82k forks source link

Error in 'topic_modeling_Gensim.ipynb' #8

Open hudmb opened 6 years ago

hudmb commented 6 years ago

Hi,

I have tried to run 'topic_modeling_Gensim.ipynb' and I get this error at this stage in the notebook. Can anyone help?: -

import random
text_data = []
with open('dataset.csv') as f:
    for line in f:
        tokens = prepare_text_for_lda(line)
        if random.random() > .99:
            print(tokens)
            text_data.append(tokens)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-54-7369a1356984> in <module>()
      3 with open('dataset.csv') as f:
      4     for line in f:
----> 5         tokens = prepare_text_for_lda(line)
      6         if random.random() > .99:
      7             print(tokens)

<ipython-input-51-4f0710beb9ee> in prepare_text_for_lda(text)
      1 def prepare_text_for_lda(text):
----> 2     tokens = tokenize(text)
      3     tokens = [token for token in tokens if len(token) > 4]
      4     tokens = [token for token in tokens if token not in en_stop]
      5     tokens = [get_lemma(token) for token in tokens]

<ipython-input-45-f5c7dc83eb04> in tokenize(text)
      3 def tokenize(text):
      4     lda_tokens = []
----> 5     tokens = parser(text)
      6     for token in tokens:
      7         if token.orth_.isspace():

NameError: name 'parser' is not defined