search

pqhieu / jsis3d

[CVPR'19] JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds
https://pqhieu.com/research/cvpr19/
MIT License
175 stars 36 forks source link

Very slow predicition #8

Closed kentaroy47 closed 4 years ago

kentaroy47 commented 4 years ago

Thanks for the great work!

While the training is fast and takes less than a second/batch, the prediction is very slow (takes 100 sec/batch.) Has anybody experienced this? I tried this on V100 instance on AWS so it shouldn't be a hardware problem.. CUDA=10.1, pytorch=1.0

It seems that the model is grabbing the GPU when I look at nvidia-smi.

pqhieu commented 4 years ago

Hi @kentaroy47,

For the prediction, we have to run mean shift clustering on the point cloud, which is entirely on CPU. Hence the prediction would be slower.

kentaroy47 commented 4 years ago

@pqhieu Thanks for the fast answers. As I was profiling, 

y = MeanShift(args['bandwidth'], n_jobs=8).fit_predict(embedded[b])

takes most of the time as you mentioned. @pqhieu do you have any tips making this faster?

pqhieu commented 4 years ago

Sorry for the late reply. It's still a bottleneck that we are trying to solve in our next work. You can increase the bandwidth size or n_jobs. Another trick is to predict the number of clusters and use k-means instead, as shown in this repo.