About how to get image to lidar dataset?

[LZL-CS]

Hi, Thanks for your nice work and shared code. When I need to get the image to correspond to the lidar point cloud, I follow your instruction to run the generate_rgb_for_lidar.py script firstly. But the code which showed as below is confused for me: May I know what are lidar2image_ndx_path, lidar2image_ndx.pickle and pickle..., etc. Do I need to run other scripts to get these files?

[jac99]

Hi,

lidar2image_ndx.pickle keeps a list of images associated with every lidar scan. This file is created by _generate_rgb_forlidar.py script (which is in scripts folder). See these lines:

# Find k closest images for each lidar timestamp
    lidar2img_ndx_pickle = 'lidar2image_ndx.pickle'
    ndx_filepath = os.path.join(out_path, 'lidar2image_ndx.pickle')
    if os.path.exists(ndx_filepath):
        with open(ndx_filepath, 'rb') as f:
            lidar2img_ndx = pickle.load(f)
    else:
        lidar2img_ndx = create_lidar2img_ndx(lidar_timestamps, image_timestamps, k, nn_threshold)
        with open(ndx_filepath, 'wb') as f:
            pickle.dump(lidar2img_ndx, f)

_lidar2imagendx.pickle should be saved in the folder given as _--outpath command line parameter when running _generate_rgb_forlidar.py script.

[LZL-CS]

Hi,

lidar2image_ndx.pickle keeps a list of images associated with every lidar scan. This file is created by _generate_rgb_forlidar.py script (which is in scripts folder). See these lines:

# Find k closest images for each lidar timestamp
    lidar2img_ndx_pickle = 'lidar2image_ndx.pickle'
    ndx_filepath = os.path.join(out_path, 'lidar2image_ndx.pickle')
    if os.path.exists(ndx_filepath):
        with open(ndx_filepath, 'rb') as f:
            lidar2img_ndx = pickle.load(f)
    else:
        lidar2img_ndx = create_lidar2img_ndx(lidar_timestamps, image_timestamps, k, nn_threshold)
        with open(ndx_filepath, 'wb') as f:
            pickle.dump(lidar2img_ndx, f)

_lidar2imagendx.pickle should be saved in the folder given as _--outpath command line parameter when running _generate_rgb_forlidar.py script.

Hi, but when I run script generate_rgb_for_lidar.py, the code run into params = MinkLocParams(args.config, model_params_path=None) first. Then, in Class MinkLocParams (inutils.py script and I choose config_baseline_multimodal.txt as config file), I have to run the code shown as below:

But I don't know how to get lidar2image_ndx.pickle for script self.lidar2image_ndx = pickle.load(open(self.lidar2image_ndx_path, 'rb')). Do I need to run other scripts before run generate_rgb_for_lidar.pyscript?

[jac99]

Yes, indeed there was a problem with the code. I've updated the code and description in the README file. Now the generation of training data should work OK. Download the updated code from github (changes were made not only in _generate_rgb_forlidar.py script but in some other files as well) and follow instructions in the updated README file. First generate training and evaluation tuples (section: Generate training and evaluation tuples). Then follow instructions in Downsample RGB images and index RGB images linked with each point cloud section. You need to edit the config file first (_config_baselinemultimodal.txt) and set _datasetfolder parameter and _imagepath (this is the path where downsampled RGB images and lidar2image_ndx.pickle will be saved). Let me know if it works now.

[LZL-CS]

Yes, indeed there was a problem with the code. I've updated the code and description in the README file. Now the generation of training data should work OK. Download the updated code from github (changes were made not only in _generate_rgb_forlidar.py script but in some other files as well) and follow instructions in the updated README file. First generate training and evaluation tuples (section: Generate training and evaluation tuples). Then follow instructions in Downsample RGB images and index RGB images linked with each point cloud section. You need to edit the config file first (_config_baselinemultimodal.txt) and set _datasetfolder parameter and _imagepath (this is the path where downsampled RGB images and lidar2image_ndx.pickle will be saved). Let me know if it works now.

It works well for me. Thanks a lot for your help, I can close this issue now.