However, I found that for QA type classification, the code includes the sub-type into the training/testing data. Needless to say it's a perfect predictor. So, I am fixing this, plus make a couple of tweaks to make it compatible with the latest Python.
I couldn't get rid of the annoying GenSim warning, but maybe you'll have more luck if you re-run the notebook.
Predictably, after removing the leak the accuracy drops. However, I verified that with ALL the data, it goes up again, at least when you randomly sample 10% from the training set to be test data. Maybe, the numbers will be different if you use the official test set. However, you data download script does not download all the data, so I kept this whole thing as is.
BTW, it takes only a few seconds to train on a MacBook pro (and a couple of minutes on all the data). Not long at all!
Hi, thank you for the great repository!
However, I found that for QA type classification, the code includes the sub-type into the training/testing data. Needless to say it's a perfect predictor. So, I am fixing this, plus make a couple of tweaks to make it compatible with the latest Python.
I couldn't get rid of the annoying GenSim warning, but maybe you'll have more luck if you re-run the notebook.
Predictably, after removing the leak the accuracy drops. However, I verified that with ALL the data, it goes up again, at least when you randomly sample 10% from the training set to be test data. Maybe, the numbers will be different if you use the official test set. However, you data download script does not download all the data, so I kept this whole thing as is.
BTW, it takes only a few seconds to train on a MacBook pro (and a couple of minutes on all the data). Not long at all!