Error compiling cudaowl

[selroc]

gpuowl (latest commit) performance on NVIDIA RTX 3090 is poor, I've attempted to compile cudaowl and got many errors:

cudaerror.txt

can you please tell me how to correct these?

[tdulcet]

What version of GCC and CUDA were you using? On Linux I was able to build it with these commands:

git clone https://github.com/preda/gpuowl.git
cd gpuowl
git checkout cuda
sed -i 's/g++ -O2/g++ -O3 -g -Wall/' Makefile
sed -i 's/-O2/-O3 -g --compiler-options=-Wall/' Makefile
make openowl
make cudaowl

and then run it like this:

echo '106928347' > worktodo.txt
./cudaowl

However, it is significantly slower than the GpuOwl master/v6 branches and even CUDALucas:

$ ./cudaowl 
2022-04-28 11:08:17 gpuowl-CUDA 3.9-9a9c3c3-mod
2022-04-28 11:08:17 Exponent 106928347 using FFT 5832K (2^13 * 3^6 * 5^0 * 7^0)
2022-04-28 11:08:18 PRP M(106928347), FFT 5832K, 17.91 bits/word, 0 GHz-day
2022-04-28 11:08:20 OK loaded: 0/106928347, blockSize 400, 0000000000000003
2022-04-28 11:08:22 OK      800/106928347 [ 0.00%], 1.59 ms/it [1.59, 1.59] (0.0 GHz-day/day); ETA 1d 23:14; 7d85dc41e3222beb (check 0.88s) (saved)
2022-04-28 11:08:37       10000/106928347 [ 0.01%], 1.59 ms/it [1.59, 1.59] (0.0 GHz-day/day); ETA 1d 23:19; c1affdcda0e4aed6
2022-04-28 11:08:53       20000/106928347 [ 0.02%], 1.59 ms/it [1.59, 1.59] (0.0 GHz-day/day); ETA 1d 23:20; 26107d02a55096b0
2022-04-28 11:09:09       30000/106928347 [ 0.03%], 1.59 ms/it [1.59, 1.60] (0.0 GHz-day/day); ETA 1d 23:21; a07da86ca96b751c
2022-04-28 11:09:25       40000/106928347 [ 0.04%], 1.60 ms/it [1.59, 1.60] (0.0 GHz-day/day); ETA 1d 23:24; dc37c5b30383e6e5
2022-04-28 11:09:41       50000/106928347 [ 0.05%], 1.60 ms/it [1.59, 1.60] (0.0 GHz-day/day); ETA 1d 23:25; ff38d7fa96e93158
2022-04-28 11:09:57       60000/106928347 [ 0.06%], 1.60 ms/it [1.60, 1.60] (0.0 GHz-day/day); ETA 1d 23:27; 1f4ccf25b1bf26f4
2022-04-28 11:10:13       70000/106928347 [ 0.07%], 1.60 ms/it [1.60, 1.60] (0.0 GHz-day/day); ETA 1d 23:29; 921912467caa2770
2022-04-28 11:10:29       80000/106928347 [ 0.07%], 1.60 ms/it [1.60, 1.60] (0.0 GHz-day/day); ETA 1d 23:29; 5ef45e7a7041ef42
2022-04-28 11:10:45       90000/106928347 [ 0.08%], 1.60 ms/it [1.60, 1.60] (0.0 GHz-day/day); ETA 1d 23:30; d37e73d9e4d9edf1
2022-04-28 11:11:01      100000/106928347 [ 0.09%], 1.60 ms/it [1.60, 1.60] (0.0 GHz-day/day); ETA 1d 23:31; 4d66b4eed5ea9ab3
^C2022-04-28 11:11:08 Stopping, please wait..
2022-04-28 11:11:09 OK   104400/106928347 [ 0.10%], 1.60 ms/it [1.60, 1.60] (0.0 GHz-day/day); ETA 1d 23:32; 359eef5c661f1f54 (check 0.88s) (saved)

This was on the Tesla V100 GPU, so you could compare the above speed to the timings listed in #253.

[selroc]

It doesn't compile for me:

$ make openowl g++ -O3 -g -Wall -DREV=\"`git rev-parse --short HEADgit diff-files --quiet || echo -mod`\" -Wall -std=c++14 OpenGpu.cpp OpenTF.cpp clwrap.cpp common.cpp gpuowl.cpp -o openowl -lOpenCL -L/opt/rocm/opencl/lib/x86_64 -L/opt/amdgpu-pro/lib/x86_64-linux-gnu -L/c/Windows/System32 common.cpp:6:8: error: aggregate ‘std::string globalCpuName’ has incomplete type and cannot be defined 6 | string globalCpuName; | ^~~~~ common.cpp: In function ‘void initLog(const char)’: common.cpp:10:22: error: invalid initialization of reference of type ‘const string&’ {aka ‘const std::__cxx11::basic_string&’} from expression of type ‘const char’ 10 | if (auto fo = open(logName, "a")) { | ^~~ In file included from common.cpp:1: common.h:54:37: note: in passing argument 1 of ‘std::unique_ptr<_IO_FILE> open(const string&, const char, bool)’ 54 | unique_ptr open(const string &name, const char mode, bool doLog = true); | ~~~~^~ common.cpp: At global scope: common.cpp:18:34: error: return type ‘std::string’ {aka ‘class std::cxx11::basic_string’} is incomplete 18 | string timeStr(const char *format) { | ^ common.cpp:25:16: error: return type ‘std::string’ {aka ‘class std::__cxx11::basic_string’} is incomplete 25 | string timeStr() { | ^ common.cpp:25:8: error: ambiguating new declaration of ‘void timeStr()’ 25 | string timeStr() { | ^~~ In file included from common.cpp:1: common.h:56:8: note: old declaration ‘std::string timeStr()’ 56 | string timeStr(); | ^~~ common.cpp:32:20: error: return type ‘std::string’ {aka ‘class std::cxx11::basic_string’} is incomplete 32 | string longTimeStr() { return timeStr("%Y-%m-%d %H:%M:%S %Z"); } | ^ common.cpp:33:21: error: return type ‘std::string’ {aka ‘class std::cxx11::basic_string’} is incomplete 33 | string shortTimeStr() { return timeStr("%Y-%m-%d %H:%M:%S"); } | ^ common.cpp: In function ‘void log(const char, ...)’: common.cpp:43:10: error: variable ‘std::string prefix’ has initializer but incomplete type 43 | string prefix = shortTimeStr() + (globalCpuName.empty() ? "" : " ") + globalCpuName; | ^~ common.cpp: In function ‘std::unique_ptr<_IO_FILE> open(const string&, const char, bool)’: common.cpp:54:33: error: invalid use of incomplete type ‘const string’ {aka ‘const class std::cxx11::basic_string’} 54 | std::unique_ptr f{fopen(name.c_str(), mode)}; | ^~~~ In file included from /usr/include/c++/11/iosfwd:39, from /usr/include/c++/11/bits/shared_ptr.h:52, from /usr/include/c++/11/memory:77, from common.h:6, from common.cpp:1: /usr/include/c++/11/bits/stringfwd.h:74:11: note: declaration of ‘std::string’ {aka ‘class std::cxx11::basic_string’} 74 | class basic_string; | ^~~~ common.cpp:54:52: error: no matching function for call to ‘std::unique_ptr<_IO_FILE>::unique_ptr()’ 54 | std::unique_ptr f{fopen(name.c_str(), mode)}; | ^ In file included from /usr/include/c++/11/memory:76, from common.h:6, from common.cpp:1: /usr/include/c++/11/bits/unique_ptr.h:350:9: note: candidate: ‘template<class _Up, class> std::unique_ptr<_Tp, _Dp>::unique_ptr(std::auto_ptr<_Up>&&) [with _Up = _Up; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 350 | unique_ptr(auto_ptr<_Up>&& u) noexcept; | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:350:9: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:340:9: note: candidate: ‘template<class _Up, class _Ep, class> std::unique_ptr<_Tp, _Dp>::unique_ptr(std::unique_ptr<_Up, _Ep>&&) [with _Up = _Up; _Ep = _Ep; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 340 | unique_ptr(unique_ptr<_Up, _Ep>&& u) noexcept | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:340:9: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:320:19: note: candidate: ‘template<class _Del, class> constexpr std::unique_ptr<_Tp, _Dp>::unique_ptr(std::nullptr_t) [with _Del = _Del; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 320 | constexpr unique_ptr(nullptr_t) noexcept | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:320:19: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:314:9: note: candidate: ‘template<class _Del, class _DelUnref> std::unique_ptr<_Tp, _Dp>::unique_ptr(std::unique_ptr<_Tp, _Dp>::pointer, std::enable_if_t<std::is_lvalue_reference<_Del>::value, _DelUnref&&>) [with _Del = _Del; _DelUnref = _DelUnref; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ (deleted) 314 | unique_ptr(pointer, | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:314:9: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:306:9: note: candidate: ‘template<class _Del, class> std::unique_ptr<_Tp, _Dp>::unique_ptr(std::unique_ptr<_Tp, _Dp>::pointer, std::enable_if_t<(! std::is_lvalue_reference<_Del>::value), _Del&&>) [with _Del = _Del; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 306 | unique_ptr(pointer p, | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:306:9: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:294:9: note: candidate: ‘template<class _Del, class> std::unique_ptr<_Tp, _Dp>::unique_ptr(std::unique_ptr<_Tp, _Dp>::pointer, const deleter_type&) [with _Del = _Del; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 294 | unique_ptr(pointer p, const deleter_type& d) noexcept | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:294:9: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:281:9: note: candidate: ‘template<class _Del, class> std::unique_ptr<_Tp, _Dp>::unique_ptr(std::unique_ptr<_Tp, _Dp>::pointer) [with _Del = _Del; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 281 | unique_ptr(pointer p) noexcept | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:281:9: note: template argument deduction/substitution failed: /usr/include/c++/11/bits/unique_ptr.h:269:19: note: candidate: ‘template<class _Del, class> constexpr std::unique_ptr<_Tp, _Dp>::unique_ptr() [with _Del = _Del; = ; _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 269 | constexpr unique_ptr() noexcept | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:269:19: note: template argument deduction/substitution failed: common.cpp:54:52: note: candidate expects 0 arguments, 1 provided 54 | std::unique_ptr f{fopen(name.c_str(), mode)}; | ^ In file included from /usr/include/c++/11/memory:76, from common.h:6, from common.cpp:1: /usr/include/c++/11/bits/unique_ptr.h:327:7: note: candidate: ‘std::unique_ptr<_Tp, _Dp>::unique_ptr(std::unique_ptr<_Tp, _Dp>&&) [with _Tp = _IO_FILE; _Dp = std::default_delete<_IO_FILE>]’ 327 | unique_ptr(unique_ptr&&) = default; | ^~~~~~ /usr/include/c++/11/bits/unique_ptr.h:327:7: note: conversion of argument 1 would be ill-formed: common.cpp:55:59: error: invalid use of incomplete type ‘const string’ {aka ‘const class std::__cxx11::basic_string’} 55 | if (!f && doLog) { log("Can't open '%s' (mode '%s')\n", name.c_str(), mode); } | ^~~~ In file included from /usr/include/c++/11/iosfwd:39, from /usr/include/c++/11/bits/shared_ptr.h:52, from /usr/include/c++/11/memory:77, from common.h:6, from common.cpp:1: /usr/include/c++/11/bits/stringfwd.h:74:11: note: declaration of ‘std::string’ {aka ‘class std::cxx11::basic_string’} 74 | class basic_string; | ^~~~ make: *** [Makefile:12: openowl] Error 1

$ make cudaowl nvcc -O3 -g --compiler-options=-Wall -DREV=\"git rev-parse --short HEAD``git diff-files --quiet || echo -mod\" -o cudaowl CudaGpu.cu NoTF.cpp common.cpp gpuowl.cpp -lcufft /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’: 435 | function(_Functor&& f) | ^ /usr/include/c++/11/bits/std_function.h:435:145: note: ‘_ArgTypes’ /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’: 530 | operator=(_Functor&& f) | ^ /usr/include/c++/11/bits/std_function.h:530:146: note: ‘_ArgTypes’ CudaGpu.h: In function ‘std::pair<std::vector, std::vector > genWeights(u32, u32)’: CudaGpu.h:26:19: warning: comparison of integer expressions of different signedness: ‘int’ and ‘u32’ {aka ‘unsigned int’} [-Wsign-compare] 26 | for (int k = 0; k < N; ++k) { | ^ make: *** [Makefile:15: cudaowl] Error 1

$ nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2021 NVIDIA Corporation Built on Thu_Nov_18_09:45:30_PST_2021 Cuda compilation tools, release 11.5, V11.5.119 Build cuda_11.5.r11.5/compiler.30672275_0 ``

[selroc]

gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0

[selroc]

can you give me your executable?

[tdulcet]

Regarding all those string errors, adding #include <string> near the top of the common.h file will fix this issue.

Regarding this error error: parameter packs not expanded with ‘...’, it looks like you would need to either upgrade to CUDA 11.6 or use an older version of GCC. See https://github.com/NVlabs/instant-ngp/issues/119 for more information.

can you give me your executable?

Unfortunately, I no longer have the executable. It was on a Google Cloud VM.

[selroc]

I have upgraded the NVIDIA CUDA Toolkit to 11.7 but still:

make cudaowl

nvcc -O3 -g --compiler-options=-Wall -DREV=\"git rev-parse --short HEAD``git diff-files --quiet || echo -mod\" -o cudaowl CudaGpu.cu NoTF.cpp common.cpp gpuowl.cpp -lcufft /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’: 435 | function(_Functor&& f) | ^ /usr/include/c++/11/bits/std_function.h:435:145: note: ‘_ArgTypes’ /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’: 530 | operator=(_Functor&& f) | ^ /usr/include/c++/11/bits/std_function.h:530:146: note: ‘_ArgTypes’ CudaGpu.h: In function ‘std::pair<std::vector, std::vector > genWeights(u32, u32)’: CudaGpu.h:26:19: warning: comparison of integer expressions of different signedness: ‘int’ and ‘u32’ {aka ‘unsigned int’} [-Wsign-compare] 26 | for (int k = 0; k < N; ++k) { | ^ make: *** [Makefile:16: cudaowl] Error 1

[tdulcet]

I would double check that you are using CUDA 11.7, as upgrading does not usually remove the old version by default. Depending on where you installed CUDA 11.7 to, you may need to run commands like these to use the new version:

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export LIBRARY_PATH=/usr/local/cuda/lib64${LIBRARY_PATH:+:${LIBRARY_PATH}}

You seem to use using the latest Ubuntu 22.04, so if the above does not work, it may help to use an older and more stable version of Ubuntu. I was using Ubuntu 20.04 on the VM. Another option would be to just build it on Google Colab.