StopIteration

[hamelena]

Hi! I applied your sentiment model to a df column, at the beginning everything worked fine but few minutes before I got the RuntimeError: generator raised StopIteration. Do you have any idea why and now to fix it? Thank you in advance.

My input is: model = FastTextSocialNetworkModel(tokenizer=tokenizer) df_clean ['sentiment'] = df_clean ['prep_text'].apply(model.predict) df_clean

What I get is:

StopIteration Traceback (most recent call last) ~/anaconda3/lib/python3.7/site-packages/razdel/segmenters/tokenize.py in segment(self, parts) 299 def segment(self, parts): --> 300 buffer = next(parts) 301 for split in parts:

StopIteration:

The above exception was the direct cause of the following exception:

RuntimeError Traceback (most recent call last)

in 8 model = FastTextSocialNetworkModel(tokenizer=tokenizer) 9 ---> 10 df_clean ['sentiment'] = df_clean ['prep_text'].apply(model.predict) 11 df_clean ~/anaconda3/lib/python3.7/site-packages/pandas/core/series.py in apply(self, func, convert_dtype, args, **kwds) 3846 else: 3847 values = self.astype(object).values -> 3848 mapped = lib.map_infer(values, f, convert=convert_dtype) 3849 3850 if len(mapped) and isinstance(mapped[0], Series): pandas/_libs/lib.pyx in pandas._libs.lib.map_infer() ~/anaconda3/lib/python3.7/site-packages/dostoevsky/models.py in predict(self, sentences, k) 82 Dict[str, float] 83 ]: ---> 84 X = self.preprocess_input(sentences) 85 Y = ( 86 self.model.predict(sentence, k=k) for sentence in X ~/anaconda3/lib/python3.7/site-packages/dostoevsky/models.py in preprocess_input(self, sentences) 76 ) 77 ) ---> 78 for sentence in sentences 79 ] 80 ~/anaconda3/lib/python3.7/site-packages/dostoevsky/models.py in (.0) 76 ) 77 ) ---> 78 for sentence in sentences 79 ] 80 ~/anaconda3/lib/python3.7/site-packages/dostoevsky/tokenization.py in split(self, text, lemmatize) 37 ]: 38 return [ ---> 39 (token.text.lower(), None) for token in regex_tokenize(text) 40 ] 41 ~/anaconda3/lib/python3.7/site-packages/dostoevsky/tokenization.py in (.0) 37 ]: 38 return [ ---> 39 (token.text.lower(), None) for token in regex_tokenize(text) 40 ] 41 ~/anaconda3/lib/python3.7/site-packages/razdel/substring.py in find_substrings(chunks, text) 16 def find_substrings(chunks, text): 17 offset = 0 ---> 18 for chunk in chunks: 19 start = text.find(chunk, offset) 20 stop = start + len(chunk) RuntimeError: generator raised StopIteration

[dveselov]

Hi, can you try to update razdel package?

$ pip install -U razdel

[hamelena]

Hi, many thanks! Now I get the error: 'float' object is not iterable

TypeError Traceback (most recent call last)

in () 8 model = FastTextSocialNetworkModel(tokenizer=tokenizer) 9 ---> 10 df['sentiment'] = df['prep_text'].apply(model.predict) 11 df /usr/lib/python3/dist-packages/pandas/core/series.py in apply(self, func, convert_dtype, args, **kwds) 2549 else: 2550 values = self.asobject -> 2551 mapped = lib.map_infer(values, f, convert=convert_dtype) 2552 2553 if len(mapped) and isinstance(mapped[0], Series): pandas/_libs/src/inference.pyx in pandas._libs.lib.map_infer() /home/***/.local/lib/python3.6/site-packages/dostoevsky/models.py in predict(self, sentences, k) 82 Dict[str, float] 83 ]: ---> 84 X = self.preprocess_input(sentences) 85 Y = ( 86 self.model.predict(sentence, k=k) for sentence in X /home/***/.local/lib/python3.6/site-packages/dostoevsky/models.py in preprocess_input(self, sentences) 76 ) 77 ) ---> 78 for sentence in sentences 79 ] 80 TypeError: 'float' object is not iterable