simonzhang00 / ripser-plusplus

Ripser++: GPU-accelerated computation of Vietoris–Rips persistence barcodes
MIT License
98 stars 14 forks source link

Illegal Memory Access when using --sparse #6

Closed nihargupte-ph closed 1 year ago

nihargupte-ph commented 3 years ago

I am using the python binding version. So some of my matrices are of size 3000+ and calculating the degree 2 homology for these matrices crashes my computer (I think I'm running out of GPU memory, I have a GTX 970). I figured I'd try to use using the --sparse feature in rpp_py.run since that would have a lower memory cost. However, when I run it I get the following message:

Failed: Cuda error /home/n/Documents/Research/etc_codes/ripser-plusplus/python/../ripser++.cu:2729 'an illegal memory access was encountered'

What is weird is if I run the same computation for a smaller matrix my computer can handle ~1000 size point cloud but don't use the --sparse option it works. But if I add the sparse option I get this error

Full error message + output


------------RIPSER++ WITH PYTHON BINDINGS CALLED------------ <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>
point cloud with 1141 points in dimension 3
value range: [0.00883786,65.4655]
sparse distance matrix with 1141 points and 497929/650370 entries
threshold: 34.8033
(sparse) max possible num simplices for memory allocation forall dims: 14187167
Failed: Cuda error /home/n/Documents/Research/etc_codes/ripser-plusplus/python/../ripser++.cu:2729 'an illegal memory access was encountered'

Any suggestions appreciated.

simonzhang00 commented 3 years ago

I don't know the data you are using so I can't know exactly what is causing the problem. You could try to debug the kernel line by line yourself. There may be an issue with the way your GPU is handling warp filtering. You could also try on a different machine of course. For more on warp filtering consult: https://developer.nvidia.com/blog/cuda-pro-tip-optimized-filtering-warp-aggregated-atomics/