search

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.23k stars 2.28k forks source link

cluster_dbscan crashes on large clouds #1600

Open szppaks opened 4 years ago

szppaks commented 4 years ago

Describe the bug I have a bigger (4088000 points) point cloud; when I run a dbscan (params:2,90,True) on it, the python crashed without any error - in the system monitor, I see it works hard on each cores (~30-40 sec), then crashes. (In spyder, the ipython console crashes without any message). I have tried to to update all (conda update --all), but it did not help.

To Reproduce load a big point cloud execute dbscan

Environment (please complete the following information):

germanros1987 commented 4 years ago

@szppaks could you provide us with the point cloud?

@griegler could you comment on this?

szppaks commented 4 years ago

Hi! On ALS point clouds it works fine. I tried on some TLS clouds. The method begins with a full cpu load, then the used memory is growing to the memory limit, and it crashes.

The attachment is a 7z file, but I have to add a .zip extension to load in github. pipes_chunk_subsam.ply.7z.zip

griegler commented 4 years ago

I just checked the point cloud with the parameters that were mentioned in the first post. The problem is that it runs out of memory. The algorithm pre-computes all the neighbors in the eps distance for each point. If eps is too big, then this list will also get quite/too big.

Further, my best guess is that the memory error happens inside of the kdtree search, and not in the std::vector, therefore, it does not throw the exception. @yxlao do you have an idea how this could be better handled?

An optional solution would be to provide a parameter to not pre-compute all neighbors. Would make the algorithm slower, but more memory efficient. @germanros1987 what do you think?