This project is considered obsolete as the Torch framework is no longer maintained. For compatibility with OpenNMT-tf or OpenNMT-py, please check out CTranslate2.
CTranslate is a C++ implementation of OpenNMT's translate.lua
script with no LuaTorch dependencies. It facilitates the use of OpenNMT models in existing products and on various platforms using Eigen as a backend.
CTranslate provides optimized CPU translation and optionally offloads matrix multiplication on a CUDA-compatible device using cuBLAS. It only supports OpenNMT models released with the release_model.lua
script.
CMake and a compiler that supports the C++11 standard are required to compile the project.
git submodule update --init
mkdir build
cd build
cmake ..
make
It will produce the dynamic library libonmt.so
(or .dylib
on Mac OS, .dll
on Windows) and the translation client cli/translate
.
CTranslate also bundles OpenNMT's Tokenizer which provides the tokenization tools lib/tokenizer/cli/tokenize
and lib/tokenizer/cli/detokenize
.
-DEIGEN3_ROOT=<path to Eigen library>
option.-DLIB_ONLY=ON
flag.-DWITH_OPENMP=OFF
flag.-DWITH_QLINEAR=AVX2|SSE
flag (OFF
by default) and set the appropriate extended instructions set via -DCMAKE_CXX_FLAGS
:
-DWITH_QLINEAR=AVX2
requires at least -mavx2
-DWITH_QLINEAR=SSE
requires at least -mssse3
-DCMAKE_CXX_FLAGS="-march=native"
to the cmake
command above to optimize for speed;cli/translate
, consider fine-tuning the level of parallelism:
--parallel
option enables concurrent translation of --batch_size
sentences--threads
option enables each translation to use multiple threads--parallel
and set --threads
to 1; if you want minimal latency for a single batch, set --parallel
to 1, and increase --threads
.See --help
on the clients to discover available options and usage. They have the same interface as their Lua counterpart.
This project is also a convenient way to load OpenNMT models and translate texts in existing software.
Here is a very simple example:
#include <iostream>
#include <onmt/onmt.h>
int main()
{
// Create a new Translator object.
auto translator = onmt::TranslatorFactory::build("enfr_model_release.t7");
// Translate a tokenized sentence.
std::cout << translator->translate("Hello world !") << std::endl;
return 0;
}
For a more advanced usage, see:
include/onmt/TranslatorFactory.h
to instantiate a new translatorinclude/onmt/ITranslator.h
(the Translator
interface) to translate sequences or batch of sequencesinclude/onmt/TranslationResult.h
to retrieve results and attention vectorsinclude/onmt/Threads.h
to programmatically control the number of threads to useAlso see the headers available in the Tokenizer that are accessible when linking against CTranslate.
CTranslate focuses on supporting model configurations that are likely to be used in production settings. It covers models trained with the default options, plus some variants:
brnn
encoder (with sum
or concat
merge policy)dot
attentionAdditionally, CTranslate misses some advanced features of translate.lua
: