search

NVlabs / tiny-cuda-nn

Lightning fast C++/CUDA neural network framework
Other
3.75k stars 454 forks source link

HashGrid Encoding benchmarking runtime error #102

Closed creiser closed 2 years ago

creiser commented 2 years ago

I can only run the benchmark with the "OneBlob" encoding but not with the "HashGrid" encoding:

$ ./build/bench_image_ours data/images/albert.jpg data/config_hash.json 
terminate called after throwing an instance of 'std::runtime_error'
  what():  /home/creiser/tiny-cuda-nn/include/tiny-cuda-nn/cuda_graph.h:103 cudaDeviceSynchronize() failed with error an illegal memory access was encountered

$ nvcc --version
Build cuda_11.1.TC455_06.29069683_0

GPU: RTX 3090 Driver Version: 510.47.03 CUDA Version: 11.6

Tom94 commented 2 years ago

Oh -- the benchmark code is basically ancient from when tiny-cuda-nn didn't have trainable encodings yet. It hasn't been properly updated, using Encoding without actually initializing its parameters (or allocating their memory for that matter).

I'm going to make a note to look into this when back from vacation. Should be as easy as replacing the manual Encoding+Network combos with NetworkWithInputEncoding objects and then initializing those by passing them to respective Trainers. Cheers!

Tom94 commented 2 years ago

Fixed on latest master via https://github.com/NVlabs/tiny-cuda-nn/commit/53f70a91f56617b1b2a7e71c17fe985bbd55b03a