search

OpenNMT / CTranslate

Lightweight C++ translator for OpenNMT Torch models (deprecated)
https://opennmt.net/
MIT License
79 stars 50 forks source link

Bad performance than translate.lua #12

Closed wsnooker closed 7 years ago

wsnooker commented 7 years ago

The performance of using CTranslate is not as good as Translate.lua. Are there any tips i missed in converting GPU model to CPU model?

Another request, I want to get the n_best and its corresponding score. But now the demo lacks these apis, do you have any plans for creating them?

guillaumekln commented 7 years ago

What is the performance difference you get between CTranslate and translate.lua? When compiled in Release mode it should as fast or faster than translate.lua on the same CPU model.

The n_best option is known feature request and there is plans to add it but no expected release time. Hopefully I can take a look at adding it this week.

wsnooker commented 7 years ago

The elapsed time is 1.5 second in compiled Release mode, while the translate.lua takes 1 second for the same sentence.

guillaumekln commented 7 years ago

How do you measure that and what is your model configuration?

wsnooker commented 7 years ago

Just translate one sentence and print the elapsed time. The model configuration is : wordvec-500, rnnHiddenSize-1000, vocabSize-50k, and just one rnn layer.

guillaumekln commented 7 years ago

Are you also measuring model loading time?

To make a fair comparison, you should duplicate your sentence like 100 or 200 times and use the -time option (--time for CTranslate) to output the average translation time.

wsnooker commented 7 years ago

Thanks, performance works fine when testing in another server machine. Maybe the server lacks sufficient memory in previous one.

hphong591992 commented 7 years ago

But if we run translate.lua with GPU, C++ translator is way slower right?

guillaumekln commented 7 years ago

Of course. GPUs are still an order of magnitude faster than CPUs.