Training time on SUN RGBD Dataset

facebookresearch / votenet

Deep Hough Voting for 3D Object Detection in Point Clouds

MIT License

1.7k stars 376 forks source link

Training time on SUN RGBD Dataset #44

Open jytime opened 5 years ago

jytime commented 5 years ago

Hi all,

I was wondering what's the training time of votenet on the sun rgbd dataset. According to my experience, using one 1080 Ti GPU on an Ubuntu 16.4 server, it would take around a day. In the paper of votenet, it's about 10 hours on a Volta Quadro GP100 GPU.

Is this difference caused by GPU version, or did I set anything wrong? I have closed the evaluation process during training. Any comment is appreciated.

Best

charlesq34 commented 5 years ago

@jytime I think your training time is reasonable. If you are using SSD as well, then I believe the GPU costs the difference.

StevenZzz07 commented 4 years ago

@jytime Hello, Recently i am also working on reproducig Votenet. I use a Windows server with one GPU 1080ti and 16GB RAM. However, when i train the net on sunrgbd dataset, I am caught with a MemoryError problem in ...\lib\multiprocessing\reduction.py and the bug line is cls(buf, protocol).dump(obj).

Since your device is simliar to mine and you have trained it successfully, i am wondering if you have done any code optimization or any other parameter modifications.

Thank you very much!!!