search

allenai / deep_qa

A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)
Apache License 2.0
404 stars 133 forks source link

Update sentence selection model to use dynamic padding #357

Closed matt-gardner closed 7 years ago

matt-gardner commented 7 years ago

Currently getting a weird error on model loading, but after that's fixed, this will hopefully just work as-is.

matt-gardner commented 7 years ago

This is now ready to review. It's updating the sentence selection model to be compatible with the new dynamic padding stuff that I've been working on. This should make training these models a lot faster. @haniesedghi, FYI.

matt-gardner commented 7 years ago

Ok, I fixed those comments, @pdasigi, and I've run some timing benchmarks. Without dynamic padding, I was truncating to 40 words per sentence. The max sentence length is ~260 words, though I don't know the average sentence length. One epoch with this truncation took about 45 minutes. When I switched to dynamic padding I removed all truncation. It was on track to finish in about 1.25 hours per epoch, but it ran out of memory on a batch with really long sentences.

If we assume a linear relationship between sentence length and runtime (which may not be accurate), we'd get maybe ~3 hours per epoch without any truncation (though it also would run out of memory, so we'd have to decrease the batch size, which would make it slower). Dynamic padding, then, gives us a ~2-3x speedup in this model.