search

Nightmare-n / GD-MAE

GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds (CVPR 2023)
Apache License 2.0
113 stars 6 forks source link

GraphVoI model perfroms worse on datasets other than KITTI #24

Open rajeev-gupta-bashrc opened 1 month ago

rajeev-gupta-bashrc commented 1 month ago

Hello, I've run the model GraphVoI on KITTI dataset and it works very good, but I also tried to see the results on other datasets. I've made some changes in the code to change the waymo dataset and nuscenes dataset format to the exactly same format as KITTI. I'm quite confident on the calibration data too that I've converted from other datasets to kitti format because I'm able to visualise images projected from the point clouds. Here have a look at the waymo dataset. The image is obtained by projecting waymo point clouds to image using calibration matrices. image and here is the original image: image

I've also noticed a thing while working on nuscenes dataset that the point cloud in that dataset is too sparse, we can't figure out any object via back projection of point clouds, and I didn't get any results in nuscenes. But waymo has approximately same density as KITTI, but still the result is an empty tensor. I've checked the point cloud range, calibration, and image size for region of interest and corresponding camera. The back projected image is proof of that.

Summary: I'll be grateful if you think about the reason for getting empty tensors in the output for the waymo dataset. In addition, for different datasets the point cloud range and voxel_size is different, the model gives some shape mismatch error if pc_range of one dataset is used in another. It will be very helpful if you share some info about it.

rajeev-gupta-bashrc commented 1 month ago

EDIT: I tried to set the prediction scores to 0.1 and now there are some predicted boxes (but ofc they are false predictions). While doing hit and trials I also set the intensity vector to 1 in the kitti dataset, and then it was giving no good results, as the same case in waymo dataset. Is it the intensity vector that is making a lot of difference while inference. Since, I couldn't extract intensity from waymo dataset I've approximated it by distance from the lidar and the height from the ground. This is the back-projected image that I'm getting from the new lidar points after creating a fake intensity vector. image

Even this image is much clear as compared to my last image, but still there isn't any result with good scores. If you can help to get intensity from waymo dataset then it would be very much appreciated, and please comment on the weightage of the point cloud intensities on the prediction results.

Nightmare-n commented 1 month ago

Hi, Graph-Ce is trained with waymo dataset. Can adding the image branch on Graph-Ce work well?

rajeev-gupta-bashrc commented 1 month ago

Hi Nightmare-n, Thank you for your reply, well I didn't try to make changes in the model, but I shall look into it as you suggested. BTW here is some updates on the issue: I followed the waymo_converter.py code in GraphRCNN and now my data has the intensity vector. I appreciate your suggestion to try GraphCE that is trained on waymo, but I also want to know, if the data-input (converted from waymo to kitti format) to GraphVoI model is similar to kitti data, then where is the issue, that I'm not getting good prediction scores. Here is the original image: image and the 2D projected image using point clouds: image Could you please help me to know, why there is no predictions with any good scores. image

Summary: If I use different models for different dataset, then I'll not be able to create a generalised model, to be used in some applications.