implement target vocabulary selection

OpenNMT / CTranslate

Lightweight C++ translator for OpenNMT Torch models (deprecated)

https://opennmt.net/

MIT License

79 stars 50 forks source link

Closed jsenellart closed 6 years ago

jsenellart commented 6 years ago

Given a source/target n-gram mapping, implement target vocabulary selection by reduction of softmax weights.
Add generic apply function to go through the graph and apply generic function
more fine-grained profiling

Initial tests on a small network with 32K vocabulary shows speedup 2.5x at the sentence level.