search

marian-nmt / marian

Fast Neural Machine Translation in C++
https://marian-nmt.github.io
Other
1.25k stars 233 forks source link

is gpu version shortlist possible? #294

Closed david-pureal closed 4 years ago

david-pureal commented 5 years ago

I have successfully try the cpu version lexical shortlist, making the decoding 2 times faster. According to the article 4.1, shortlist is only for cpu-bound accelaration. My question is how about the gpu version? Will marian implement it on schedule? Thanks!

snukky commented 5 years ago

Yes, lexical shortlists should work for both CPU and GPU decoding. It just brings a smaller improvement on GPUs.

(The article mentioned in the question: https://www.aclweb.org/anthology/W18-2716.pdf)

david-pureal commented 5 years ago

Yes, lexical shortlists should work for both CPU and GPU decoding. It just brings a smaller improvement on GPUs.

(The article mentioned in the question: https://www.aclweb.org/anthology/W18-2716.pdf)

Thanks for your reply. I have read the paper, but there is no GPU shortlist experiment mentioned in it. It only has CPU shortlist, have you tried the GPU shortlist?

snukky commented 5 years ago

I have tried, but do not have exact numbers now.

david-pureal commented 5 years ago

cool, I have read the src code of cpu shortlist, could you give me some hints to implement the gpu version?

snukky commented 4 years ago

Lexical shortlists on GPUs have been available since its CPU version is available.