search

ottokart / punctuator2

A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text
http://bark.phon.ioc.ee/punctuator
MIT License
657 stars 195 forks source link

Punctuate with GPU #33

Open MeteorBurn opened 6 years ago

MeteorBurn commented 6 years ago

Hello! I try to use GPU with CUDA with cuDNN for punctuating on Windows. My Theano tests show that CPU (i7 7700) spend 11 sec and GPU (GTX 1060 6 GB) 0,2 sec.

But I noticed interesting thing, when I punctuate a big text with more 75 000 symbols, GPU punctuates texts faster than CPU, but when text has less 50 000 symbols, then CPU punctuates texts faster than GPU.

How can this be? And how to make the GPU punctuate texts faster regardless of their size?

robinsax commented 6 years ago

There's some overhead involved in loading to and from the GPU so it makes sense you would see a threshold like that. Not sure how to get around that, it's more of a hardware thing.

MeteorBurn commented 6 years ago

Thank you!