The version ('80100') of the host compiler ('Apple clang') is not supported

[loretoparisi]

The cmake .. was successfull:

[loretoparisi@:mbploreto build]$ cmake ..
-- Build type: Release
-- Try OpenMP C flag = [ ]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-fopenmp=libomp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-Qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-xopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [+Oopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-qsmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP C flag = [-mp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [ ]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-fopenmp=libomp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [/openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-Qopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-openmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-xopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [+Oopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-qsmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Try OpenMP CXX flag = [-mp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Failed
-- Could NOT find OpenMP (missing:  OpenMP_C_FLAGS OpenMP_CXX_FLAGS) 
CMake Warning at CMakeLists.txt:22 (message):
  OpenMP not found: Compilation will not use OpenMP

-- Boost version: 1.64.0
-- Found the following Boost libraries:
--   program_options
-- Boost version: 1.64.0
-- Found the following Boost libraries:
--   program_options
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/loretoparisi/Documents/Projects/AI/CTranslate/build

while when doing make:

[loretoparisi@:mbploreto build]$ make
Scanning dependencies of target OpenNMTTokenizer
[  3%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/BPE.cc.o
[  6%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/CaseModifier.cc.o
[  9%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/ITokenizer.cc.o
[ 12%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/SpaceTokenizer.cc.o
[ 15%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/Tokenizer.cc.o
[ 18%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/unicode/Data.cc.o
[ 21%] Building CXX object lib/tokenizer/CMakeFiles/OpenNMTTokenizer.dir/src/unicode/Unicode.cc.o
[ 24%] Linking CXX shared library libOpenNMTTokenizer.dylib
[ 24%] Built target OpenNMTTokenizer
Scanning dependencies of target TH
[ 27%] Building C object lib/TH/CMakeFiles/TH.dir/THGeneral.c.o
[ 30%] Building C object lib/TH/CMakeFiles/TH.dir/THFile.c.o
[ 33%] Building C object lib/TH/CMakeFiles/TH.dir/THDiskFile.c.o
[ 36%] Linking C shared library libTH.dylib
[ 36%] Built target TH
[ 39%] Building NVCC (Device) object CMakeFiles/onmt.dir/src/cuda/onmt_generated_Kernels.cu.o
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc fatal   : The version ('80100') of the host compiler ('Apple clang') is not supported
CMake Error at onmt_generated_Kernels.cu.o.Release.cmake:222 (message):
  Error generating
  /Users/loretoparisi/Documents/Projects/AI/CTranslate/build/CMakeFiles/onmt.dir/src/cuda/./onmt_generated_Kernels.cu.o

make[2]: *** [CMakeFiles/onmt.dir/src/cuda/onmt_generated_Kernels.cu.o] Error 1
make[1]: *** [CMakeFiles/onmt.dir/all] Error 2
make: *** [all] Error 2

[guillaumekln]

nvcc fatal : The version ('80100') of the host compiler ('Apple clang') is not supported

There is an incompatibility between your CUDA version and compiler version. You should find a workaround by just Googling the error The version ('80100') of the host compiler ('Apple clang') is not supported.

If you do not care about CUDA support, you can also disable it:

rm -f CMakeCache.txt
cmake -DCUDA_TOOLKIT_ROOT_DIR="" ..

[loretoparisi]

@guillaumekln thank you, this time it worked:

[ 93%] Building CXX object cli/CMakeFiles/translate.dir/BatchReader.cc.o
[ 96%] Building CXX object cli/CMakeFiles/translate.dir/BatchWriter.cc.o
[100%] Linking CXX executable translate
[100%] Built target translate

and

[loretoparisi@:mbploreto cli]$ ./translate --help
OpenNMT Translator:
  --help                       display available options
  --model arg                  path to the OpenNMT model
  --src arg                    path to the file to translate (read from the 
                               standard input if not set)
  --tgt arg                    path to the output file (write to the standard 
                               output if not set
  --phrase_table arg           path to the phrase table
  --replace_unk                replace unknown tokens by source tokens with the
                               highest attention
  --batch_size arg (=30)       batch size
  --beam_size arg (=5)         beam size
  --max_sent_length arg (=250) maximum sentence length to produce
  --time                       output average translation time
  --profiler                   output per module computation time
  --threads arg (=0)           number of threads to use (set to 0 to use the 
                               number defined by OpenMP)
  --cuda   

Let me check it (Tokenize question :) before closing this.

[loretoparisi]

I'm trying to read from stdin and write to stdout like, but I get a Segmentation fault:

[loretoparisi@:mbploreto cli]$ ./translate --model /root/wmt14.en-fr.fconv-float/model.th7 --beam_size 5
Segmentation fault: 11

[guillaumekln]

Is it a model trained with OpenNMT?

[loretoparisi]

@guillaumekln Ah! that was the issue, it's a Fairseq pretrained model! I guess they both had a similar LSTM. I'm going to download from http://opennmt.net/Models/, and try again. Give me a while, thank you.

[loretoparisi]

@guillaumekln ok, thanks now it seems it is working when feeding from stdin

[loretoparisi@:mbploreto cli]$ echo "The quick brown fox jumps over the lazy dog" |  ./translate --model /root/onmt_baseline_wmt15-all.en-de_epoch13_7.19_release.t7 --beam_size 5
Der <unk> Fuchs springt über den faulen Hund

Are the <unk> tokens not available in this pretrained dict (in this case quick)? Is the translate applying the BPE tokenization or I have to use tokenize before it?

[guillaumekln]

Are the tokens not available in this pretrained dict (in this case quick)?

Yes, <unk> are out of vocabulary words. This model generates them often as the target is non subtokenized German.

Is the translate applying the BPE tokenization or I have to use tokenize before it?

It expects space-separated tokens after tokenization.

[loretoparisi]

@guillaumekln Thank you. Okay, so I have to tokenize with tokenize cli, before calling translate. Is this way correct:

[loretoparisi@:mbploreto build]$  echo "The quick brown fox jumps over the lazy dog" | \
 ./lib/tokenizer/cli/tokenize | \
./cli/translate --model /root/onmt_baseline_wmt15-all.en-de_epoch13_7.19_release.t7 --beam_size 5

[guillaumekln]

Yes. More generally, you should apply the same tokenization you used on your training data.

[loretoparisi]

@guillaumekln ok clear that. So considering these pre-trained, are them tokenized with BPE?

[guillaumekln]

It's described in the "Corpus Prep" column.

[loretoparisi]

Got it, thanks. In this case it would be a aggressive mode then!

echo "The quick brown fox jumps over the lazy dog" | \
./lib/tokenizer/cli/tokenize --mode aggressive | \
./cli/translate --model /root/onmt_baseline_wmt15-all.en-de_epoch13_7.19_release.t7 --beam_size 5
Der <unk> Fuchs springt über den faulen Hund

while the multilingual model it fails:

echo "Le renard brun rapide saute sur le chien paresseux" | ./lib/tokenizer/cli/tokenize --mode aggressive | ./cli/translate --model /root/onmt_esfritptro-4-1000-600_epoch13_3.12_release.t7 --beam_size 5
$ Error: Assertion `thtensor' failed.  at /Users/loretoparisi/Documents/Projects/AI/CTranslate/src/th/Obj.cc:378

but it could be a GPU trained model.

[loretoparisi]

@guillaumekln Was the Error: Assertionthtensor' failed.` due to the GPU/CPU model? Thanks, I'm closing this then.

[guillaumekln]

Yes, because it was a GPU model.

[loretoparisi]

Ok thank you gonna closing this one.