Reason for MAX_SENTENCE_LEN = 128

AMontgomerie / question_generator

An NLP system for generating reading comprehension questions

MIT License

281 stars 72 forks source link

Reason for MAX_SENTENCE_LEN = 128 #5

Closed PS-AI closed 2 years ago

PS-AI commented 3 years ago

Hi, Thank you for question_generator. I am a beginner and am trying to understand the code and use it for multiple choice question generation. I am unable to understand why you have restricted sentence length to 128 characters in the function _split_text(self, text). Could you clarify?

AMontgomerie commented 3 years ago

Hi, sorry I never responded to this. Somehow I didn't get a notification about it.

As I recall, I just wanted to split the text into short segments which could be used as answers. 128 is an arbitrary number I chose to be the maximum length. I found that if I didn't limit the length of the sentences, some of the answers were long complex sentences which the system was unable to generate coherent questions for.