search

bloomsburyai / question-generation

Neural text-to-text question generation
MIT License
217 stars 52 forks source link

TypeError : Nonetype object not iterable #41

Closed ayush2051 closed 4 years ago

ayush2051 commented 4 years ago

I have been getting a typerror Nonetype object not iterable in the below line, ctxts[i], ans_pos[i] = preprocessing.filter_context(ctxts[i], ans_pos[i], filter_window_size_before, filter_window_size_after, filter_max_tokens)

After having populated the answers & passing onto the generation model batch-wise, it's showing the error, "Couldn't find the char position in the filter_context block of preprocessing"

tomhosking commented 4 years ago

Hi, what dataset are you using? The error implies that the answer string couldn't be found in the context.

ayush2051 commented 4 years ago

Hi Tom, I have used a custom dataset where I have generated custom answers which are then fed to the generation model. In one of the contexts, it is failing even if the answer heads are present inside the context. Unable to figure out the issue why it is failing.

tomhosking commented 4 years ago

Does the context or answer contain any non ascii characters? The tokenizer can struggle with tracking character offsets for dual-width characters.