xinyadu / nqg

neural question generation for reading comprehension
https://arxiv.org/abs/1705.00106
MIT License
345 stars 103 forks source link

About word tokenizing and sentence splitting #11

Open Chen-Wang-CUHK opened 6 years ago

Chen-Wang-CUHK commented 6 years ago

Many thanks for your kind code sharing!

Could you provide the code to preprocess the data? Or Could you give us your configurations to use CoreNLP?

Thanks again!

xinyadu commented 6 years ago

Hello,

I was using the python wrapper for CoreNLP for data processing: https://github.com/smilli/py-corenlp

roshansridhar commented 5 years ago

Hello, I was wondering if you could tell us what tokens you used. For example I notice you replace original with the new tags for the following: ( to -lrb-, ) to -rrb-, - to --, etc