This CTranslate2 package was not compiled with CUDA support

[ciayomin]

when i use ctranlate2 on AArch64， it throws an error: This CTranslate2 package was not compiled with CUDA support, how can i solved it?

[guillaumekln]

Currently you need to compile from the sources in order to enable CUDA on a AArch64 system.

Is it a NVIDIA Jetson platform?

[skye-repos]

I've been trying to build it on my AMD system so that i can use whisperX with cuda support (whisper works with Cuda meaning faster-whisper ==> ctranslate2) is the issue since it wasn't compiled with cuda support.

I get this

/usr/include/bits/mathcalls.h(110): error: identifier "_Float32" is undefined
  extern _Float32 modff32 (_Float32 __x, _Float32 *__iptr) noexcept (true); extern _Float32 __modff32 (_Float32 __x, _Float32 *__iptr) noexcept (true) __attribute__ ((__nonnull__ (2)));
                                                                                   ^

/usr/include/bits/mathcalls.h(110): error: identifier "_Float32" is undefined
  extern _Float32 modff32 (_Float32 __x, _Float32 *__iptr) noexcept (true); extern _Float32 __modff32 (_Float32 __x, _Float32 *__iptr) noexcept (true) __attribute__ ((__nonnull__ (2)));
                                                                                                       ^

Error limit reached.
100 errors detected in the compilation of "/home/skye/Projects/whisperX/CTranslate2/src/cuda/random.cu".
Compilation terminated.
CMake Error at ctranslate2_generated_random.cu.o.Release.cmake:280 (message):
  Error generating file
  /home/skye/Projects/whisperX/CTranslate2/build/CMakeFiles/ctranslate2.dir/src/cuda/./ctranslate2_generated_random.cu.o

make[2]: *** [CMakeFiles/ctranslate2.dir/build.make:1696: CMakeFiles/ctranslate2.dir/src/cuda/ctranslate2_generated_random.cu.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/ctranslate2.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

any advice? I set the c compiler to gcc12 cause the internet suggested it, but that didn't work.

[guillaumekln]

The released binaries for x64 (Intel/AMD) are already compiled with CUDA support. Why do you need to compile again?

[wgimperial]

Currently you need to compile from the sources in order to enable CUDA on a AArch64 system.

Is it a NVIDIA Jetson platform?

Is the Dockerfile suppport on AArch64?

[ILC1997]

May I ask how to start CUDA on AArch64 and compile from source code? I have been trying for several days but still failed. Can you teach me how to complete this work? The more detailed, the better. thanks

[guillaumekln]

Is the Dockerfile suppport on AArch64?

No. See this other issue #1155.

May I ask how to start CUDA on AArch64 and compile from source code? I have been trying for several days but still failed. Can you teach me how to complete this work? The more detailed, the better. thanks

I never tried compiling with CUDA on AArch64 so I don't have specific instructions to share.

[Geremia]

I get hundreds of these sorts of errors:

/usr/include/bits/mathcalls.h(101): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(104): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(107): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(110): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(53): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(55): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(57): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(59): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(62): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(64): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(66): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(71): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(73): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(75): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(79): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(85): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(87): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(89): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(95): error: identifier "_Float32" is undefined
/usr/include/bits/mathcalls.h(98): error: identifier "_Float32" is undefined
/usr/include/stdlib.h(141): error: identifier "_Float32" is undefined
/usr/include/stdlib.h(147): error: identifier "_Float64" is undefined
/usr/include/stdlib.h(153): error: identifier "_Float128" is undefined
/usr/include/stdlib.h(159): error: identifier "_Float32x" is undefined
/usr/include/stdlib.h(165): error: identifier "_Float64x" is undefined
/usr/include/stdlib.h(234): error: identifier "_Float32" is undefined
/usr/include/stdlib.h(240): error: identifier "_Float64" is undefined
/usr/include/stdlib.h(246): error: identifier "_Float128" is undefined
/usr/include/stdlib.h(252): error: identifier "_Float32x" is undefined
/usr/include/stdlib.h(258): error: identifier "_Float64x" is undefined
/usr/include/stdlib.h(317): error: identifier "_Float32" is undefined
/usr/include/stdlib.h(324): error: identifier "_Float64" is undefined
/usr/include/stdlib.h(331): error: identifier "_Float128" is undefined
/usr/include/stdlib.h(338): error: identifier "_Float32x" is undefined
/usr/include/stdlib.h(345): error: identifier "_Float64x" is undefined

Error limit reached.
100 errors detected in the compilation of "/tmp/CTranslate2/src/ops/topp_mask_gpu.cu".
Compilation terminated.
CMake Error at ctranslate2_generated_topp_mask_gpu.cu.o.Release.cmake:280 (message):
  Error generating file
  /tmp/usr/CTranslate2/build/CMakeFiles/ctranslate2.dir/src/ops/./ctranslate2_generated_topp_mask_gpu.cu.o

make[2]: *** [CMakeFiles/ctranslate2.dir/build.make:14853: CMakeFiles/ctranslate2.dir/src/ops/ctranslate2_generated_topp_mask_gpu.cu.o] Fehler 1
make[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/ctranslate2.dir/all] Fehler 2
make: *** [Makefile:136: all] Fehler 2

CUDA 12.2.0

[Geremia]

@guillaumekln

The released binaries for x64 (Intel/AMD) are already compiled with CUDA support. Why do you need to compile again?

I want to compile against CUDA 12 (not CUDA 11 like the prebuilt binaries).

[guillaumekln]

This error is also reported here and the reason is the GCC version that is too recent. Make sure the host compiler version is supported by CUDA:

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#host-compiler-support-policy

[Geremia]

@guillaumekln Maybe. I'm using GCC 13.1. I tried make CC=gcc-5 CPP=g++-5 CXX=g++-5 LD=g++-5 but get the same errors.

When running Cmake, I got this error:

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_LIBRARY (ADVANCED)
    linked by target "ctranslate2" in directory

[Geremia]

@guillaumekln Is there a workaround that doesn't involve downgrading GCC?

[ILC1997]

@guillaumekln Thank you, may I ask? I am executing the cmka .. in the official document When commanding, an error message appears as: CMake Error at CMakeLists. txt: 300 (message): MKL include directory not found

Then I failed to install MKL, can you tell me why? Is it because the ARM platform does not support MKL

[Geremia]

@ILC1997 That seems unrelated to this issue. Use -DWITH_MKL=0.

[guillaumekln]

@Geremia You could build in a Docker container if you can't easily install some development packages. There are official development images from NVIDIA:

https://gitlab.com/nvidia/container-images/cuda/-/blob/master/doc/supported-tags.md

[Geremia]

@guillaumekln Master works for me, even with CUDA 12.2.91 & GCC 11.2.0 (in accordance with your link above).

This worked:

cmake .. -DWITH_CUDA=ON -DWITH_CUDNN=ON -DWITH_MKL=OFF -DCMAKE_CXX_COMPILER=/tmp/gcc-11/usr/bin/g++-gcc-11.2.0 -DCMAKE_C_COMPILER=/tmp/gcc-11/usr/bin/gcc-11.2.0

Where I decompressed the Slack packages gcc-11.2.0-x86_64-2.txz and gcc-g++-11.2.0-x86_64-2.txz into the same directory.