search

src-d / kmcuda

Large scale K-means and K-nn implementation on NVIDIA GPU / CUDA
Other
791 stars 145 forks source link

Space requirements #26

Open mpetri opened 6 years ago

mpetri commented 6 years ago

Could you elaborate on the space requirements on GPU. I'm trying to run a 2.9Mx300 with K=32000 instance on a 16GB RAM P100 GPU and I'm getting out of memory errors. This instance should be smaller than what you have described in the readme with 8Mx256 and K=1024. Here the output:

arguments: 1 (nil) 0.000 0.10 0 2953377 300 32000 1234 1 0 2 0x7f72854f7010 0x7f727f453010 0xa16d2a0 0x7fff3ebcc0dc
reassignments threshold: 0
yinyang groups: 3200
cudaMalloc(&__ptr, __size)
/home/563/mp9691/kmcuda/src/kmcuda.cc:456 -> out of memory
failed to allocate 9453759777 bytes for device_bounds_yy
Status: MemoryAllocationFailure
vmarkovtsev commented 6 years ago

I see from the log that yinyang ratio is set to 0.1, this means that it tries to allocate 2.9Mx2.9Mx0.1 elements and obviously fails at some point. I recommend to disable yinyang in your case by explicitly setting yinyang_t to 0.

mpetri commented 6 years ago

Ok thanks. I wasn't aware that the memory usage is tied to this parameter. Maybe a note in the readme would be good?

vmarkovtsev commented 6 years ago

Indeed - I will. Thanks.