search

V2AI / Det3D

World's first general purpose 3D object detection codebse.
https://arxiv.org/abs/1908.09492
Apache License 2.0
1.49k stars 298 forks source link

cudahash: Completely failed to build Cuda error in file #48

Closed ZaiwangLi closed 4 years ago

ZaiwangLi commented 4 years ago

During training, I meet the problem below at epoch 63 with --nproc_per_node=2, samples_per_gpu=6 and workers_per_gpu=6 at epoch 81 with --nproc_per_node=2, samples_per_gpu=4 and workers_per_gpu=4

cudahash: Completely failed to build Cuda error in file '/root/spconv/src/cuhash/hash_table.cpp' in line 194 : an illegal memory access was encountered.

poodarchu commented 4 years ago

Please use template

peiyunh commented 4 years ago

I suspect you might be using spconv-1.1. In that case, you need to add use_hash=False as an additional argument every time you instantiate a sparse convolution operation (or sub-manifold sparse convolution) in scn.py. An easier way out is to follow instructions in INSTALL.md and checkout a specific branch.

ZaiwangLi commented 4 years ago

@peiyunh Thanks for reply. Hey guys, I haven't try @peiyunh suggestion.

I solved this problem by this answer: [https://github.com/traveller59/spconv/issues/74]

poodarchu commented 4 years ago

@peiyunh Thanks for reply. Hey guys, I haven't try @peiyunh suggestion.

I solved this problem by this answer: [https://github.com/[traveller59/spconv/issues/74](https://github.com/traveller59/spconv/issues/74)]

I suggest you use my version of spconv.