Something is wrong with the mapping file lidar2image_ndx.pickle

[JackFrost67]

I don't know if it's me, but I am using it to map lidar to images of the oxford dataset and I end up with a very low number of samples per run. I supposed the mapping doesn't map all the point clouds to an image.

[jac99]

Hi, lidar2image_ndx.pickle contains point cloud-to-image mapping only for point clouds present in the dataset prepared by authors of PointNetVLAD paper. These point clouds were constructed by accumulating raw 2D lidar scans (from Oxford RobotCar dataset) during the 20 meters drive. As far as I remember, there should be between 300 and 400 point clouds per run/traversal. lidar2image_ndx.pickle doesn't contain mapping of the 'original' point cloud in Oxford RobotCar dataset.

[JackFrost67]

2 questions:

1. is there a difference between the dataset downsampled by the author of Pointnetvlad and yours? (for clarity pointnetvlad and yours)
2. in the case there are no differences, is the file a full map of the pcl in the dataset? cause if u check file-wise the presence of a key in the dictionary, the key 1400505900453981 of the pcl 1400505900453981.bin is not present

[jac99]

Yes, we used exactly the same set of point clouds as in Pointnetvlad paper. I've checked, and indeed point cloud with a key 400505900453981 is missing from the lidar2image_ndx.pickle. It turns out that this point cloud (and some more) are not part of the training and evaluation set (we used the same training and evaluation as in PointNetVlad paper). They were used in the validation set. But by mistake, the line defining the validation set was removed from the configuration file (from config_baseline_multimodal.txt). So, the script to generate lidar2image_ndx.pickle indexed only point clouds in the training and evaluation subset. I've added the line to use a validation set to the configuration file in the repo (line val_file = test_queries_baseline.pickle was added to config_baseline_multimodal.txt in the config folder). Now you can re-generate the lidar2image_ndx.pickle index using renerate_rgb_for_lidar.py script. Now, it'll include point clouds from the train, evaluation, and validation sets. I'll also try to upload a zip with a re-generated index and downsampled RGB images to the cloud drive and post it in the readme. But it'll take a few days.

[JackFrost67]

didn't see the error in the configuration file. I will wait for the re-upload

[JackFrost67]

I have to reopen this issue. I downloaded the stereo-centre dataset and run the generate_rbg_for_lidar.py script. Everything looks okay, but checking the keys in the lidar2image_ndx.pickle file, I can't find some keys like 1447412864415205 Is there any reason why some keys might not be mapped to any image?

[jac99]

Hi, the script generate_rgb_for_lidar.py finds and indexes close images only for point clouds that are included in: training file (training_queries_baseline.pickle), validation (test_queries_baseline.pickle) or any of the evaluation files (oxford_evaluation_database.pickle, oxford_evaluation_query.pickle). I suspect this point cloud (1447412864415205) is not within one of these files, thus it's not indexed. But i don't know why it's not included in any of these files.

[jac99]

Closing