search

marian-nmt / marian

Fast Neural Machine Translation in C++
https://marian-nmt.github.io
Other
1.21k stars 227 forks source link

What versions of CUDA & GCC are required to compile Marian-nmt? #414

Open Edward205 opened 1 year ago

Edward205 commented 1 year ago

I attempted to compile marian-nmt on my Arch installation, which is using GCC 13.1.1 and CUDA 12.1. However, the compilation process failed. I followed these steps:

  1. Cloned the marian-nmt repository into a folder named "marian" on my desktop.
  2. Tried to install marian-nmt using the instructions provided, but encountered errors. Compilation process log

Unfortunately, I'm unable to downgrade GCC on my Arch installation. As a workaround, I created a new partition and installed Ubuntu 18.04. However, I encountered issues with PGP signing of the CUDA 10 packages during the installation.

I consulted the compatibility table available at https://github.com/marian-nmt/marian-dev/issues/526 and discovered that I'm unable to install any CUDA or GCC version that matches the tested configurations.

Could you please provide information on the latest tested versions of GCC and CUDA that are known to work with marian-nmt? Thank you.

snukky commented 1 year ago

Maybe this one https://gist.github.com/ax3l/9489132? It seems CUDA 12 doesn't support GCC version higher than 12.2.1.

Edward205 commented 1 year ago

@snukky Downgrading to GCC 12.2.1 and retrying to compile, it did get further but failed with these errors:

[ 58%] Building CXX object src/CMakeFiles/marian_cuda.dir/tensors/gpu/prod.cpp.o
[ 59%] Building CXX object src/CMakeFiles/marian_cuda.dir/tensors/gpu/prod_sparse.cpp.o
In file included from /home/edward/Desktop/marian/marian/src/common/definitions.h:5,
                 from /home/edward/Desktop/marian/marian/src/tensors/tensor.h:3,
                 from /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.h:3,
                 from /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:9:
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTyped(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = float; ComputeType = float]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:449:1:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:24:23: error: pointer used after ‘void operator delete(void*, std::size_t)’ [-Werror=use-after-free]
   24 |     if(x != 0 && --x->references_ == 0) {    \
      |                    ~~~^~~~~~~~~~~
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTyped(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = float; ComputeType = float]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:448:18:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:25:14: note: call to ‘void operator delete(void*, std::size_t)’ here
   25 |       delete x;                              \
      |              ^
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTypedLegacy(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = float; ComputeType = float]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:550:1:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:24:23: error: pointer used after ‘void operator delete(void*, std::size_t)’ [-Werror=use-after-free]
   24 |     if(x != 0 && --x->references_ == 0) {    \
      |                    ~~~^~~~~~~~~~~
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTypedLegacy(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = float; ComputeType = float]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:549:18:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:25:14: note: call to ‘void operator delete(void*, std::size_t)’ here
   25 |       delete x;                              \
      |              ^
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTypedLegacy(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = __half; ComputeType = float]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:550:1:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:24:23: error: pointer used after ‘void operator delete(void*, std::size_t)’ [-Werror=use-after-free]
   24 |     if(x != 0 && --x->references_ == 0) {    \
      |                    ~~~^~~~~~~~~~~
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTypedLegacy(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = __half; ComputeType = float]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:549:18:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:25:14: note: call to ‘void operator delete(void*, std::size_t)’ here
   25 |       delete x;                              \
      |              ^
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTyped(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = __half; ComputeType = __half]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:449:1:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:24:23: error: pointer used after ‘void operator delete(void*, std::size_t)’ [-Werror=use-after-free]
   24 |     if(x != 0 && --x->references_ == 0) {    \
      |                    ~~~^~~~~~~~~~~
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
In function ‘void marian::intrusivePtrRelease(MemoryPiece*)’,
    inlined from ‘IntrusivePtr<T>::~IntrusivePtr() [with T = marian::MemoryPiece]’ at /home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:66:26,
    inlined from ‘void marian::gpu::ProdBatchedTyped(marian::Tensor, marian::Ptr<marian::Allocator>, marian::Tensor, marian::Tensor, bool, bool, ComputeType, ComputeType) [with ElementType = __half; ComputeType = __half]’ at /home/edward/Desktop/marian/marian/src/tensors/gpu/prod.cpp:448:18:
/home/edward/Desktop/marian/marian/src/common/intrusive_ptr.h:25:14: note: call to ‘void operator delete(void*, std::size_t)’ here
   25 |       delete x;                              \
      |              ^
/home/edward/Desktop/marian/marian/src/tensors/memory_piece.h:14:3: note: in expansion of macro ‘ENABLE_INTRUSIVE_PTR’
   14 |   ENABLE_INTRUSIVE_PTR(MemoryPiece)
      |   ^~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [src/CMakeFiles/marian_cuda.dir/build.make:160: src/CMakeFiles/marian_cuda.dir/tensors/gpu/prod.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:392: src/CMakeFiles/marian_cuda.dir/all] Error 2
make: *** [Makefile:156: all] Error 2
antot commented 4 months ago

I got the same error compiling Marian (v 1.12.0) with GCC 12.2, using Boost/1.81.0-GCC-12.2.0, CMake/3.24.3-GCCcore-12.2.0, CUDA/12.1.1: Software/git/marian_1_12_0/src/common/intrusive_ptr.h:24:23: error: pointer used after ‘void operator delete(void*, std::sizet)’ [-Werror=use-after-free] 24 | if(x != 0 && --x->references == 0) {

However, it compilled successfully with GCC 11.3, using Boost/1.79.0-GCC-11.3.0, CMake/3.24.3-GCCcore-11.3.0, CUDA/12.1.1