harrisonvanderbyl / rwkv-cpp-accelerated

A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependencies
MIT License
306 stars 19 forks source link

Failed to run `examples/storygen/amd.sh` on ROCm 5.6 #35

Open marty1885 opened 1 year ago

marty1885 commented 1 year ago

Hi, I'm trying to run this repo on my AMD card. I have HIP and ROCm running. But fail to run the example provided in README. Am I doing anything wrong?

❯ ./amd.sh
In file included from ./storygen.cpp:1:
In file included from ../../include/rwkv.h:1:
../../include/rwkv/rwkv/rwkv.h:249:21: error: reference to __device__ function 'operator delete[]' in __host__ function
    int **tensors = new int *[46];
                    ^
/opt/rocm/llvm/lib/clang/16.0.0/include/cuda_wrappers/new:73:24: note: 'operator delete[]' declared here
__device__ inline void operator delete[](void* ptr) CUDA_NOEXCEPT {
                       ^
In file included from ./storygen.cpp:1:
In file included from ../../include/rwkv.h:1:
../../include/rwkv/rwkv/rwkv.h:249:21: error: reference to __device__ function 'operator delete[]' in __host__ function
    int **tensors = new int *[46];
                    ^
./storygen.cpp:11:17: note: called by 'main'
    RWKV Rwkv = RWKV();
                ^
/opt/rocm/llvm/lib/clang/16.0.0/include/cuda_wrappers/new:73:24: note: 'operator delete[]' declared here
__device__ inline void operator delete[](void* ptr) CUDA_NOEXCEPT {
                       ^
2 errors generated when compiling for host.

System info

OS: Arch Linux x64 HIPCC: 5.6.31061 C++ compiler: GCC 13

❯ hipcc --version
HIP version: 5.6.31061-
clang version 16.0.0
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm/llvm/bin

Thanks

harrisonvanderbyl commented 1 year ago

You may need to specify c++17 somewhere. Please also make sure to edit ./amd.sh to set your hardware model