search

ucaszyp / STEPS

This is the official repository for ICRA-2023 paper "STEPS: Joint Self-supervised Nighttime Image Enhancement and Depth Estimation"
https://arxiv.org/abs/2302.01334
MIT License
175 stars 10 forks source link

How do you pre-process the Robotcar dataset #1

Closed goxq closed 1 year ago

goxq commented 1 year ago

Thanks for your excellent work!

Since the image in the robotcar dataset is distorted(and in a gray tone) and is in a full-size, I wonder how you specifically pre-process the images in this dataset?

ucaszyp commented 1 year ago

Thanks for your excellent work!

Since the image in the robotcar dataset is distorted(and in a gray tone) and is in a full-size, I wonder how you specifically pre-process the images in this dataset?

Thanks for your excellent work!

Since the image in the robotcar dataset is distorted(and in a gray tone) and is in a full-size, I wonder how you specifically pre-process the images in this dataset?

Thanks for your interests! We use the official python-sdk of RobotCar (https://github.com/ori-mrg/robotcar-dataset-sdk), which provides project_laser_into_image, demosaic and undistort function. You can easily find them in project_laser_into_camer.py and image.py. Args we used are shown in the following figure. Under this setting, input Bayer images are transformed to RGB by demasaicing function(image.py line 46) and then undistorted(image.py line 48). Finally, pointcloud is projected into the image (project_laser_into_camer.py) Note that, the number of LiDAR points in one frame is small, so the official python-sdk combines multiple frames to generate ground truth depth based on Structure-from-Motion. Moving objects lead to wrong outputs. Therefore, the previous works manually select scenes without moving objects. In addition, we follow the previous work which center crops the full-size image to 1152 640 and then resizes it to 576 320. args

goxq commented 1 year ago

@ucaszyp Thank you so much for the very helpful information!!!

So the conversion and undistorted of the images is done in advance (results are placed in the 'rgb' folder) and the cropping and resizing of the images is done when the dataset is loaded (in the code), is that correct?

I'd also like to confirm what the value of 'intrinsic.npy' is in the robot car dataset.

Thx!!

ucaszyp commented 1 year ago

@ucaszyp Thank you so much for the very helpful information!!!

So the conversion and undistorted of the images is done in advance (results are placed in the 'rgb' folder) and the cropping and resizing of the images is done when the dataset is loaded (in the code), is that correct?

I'd also like to confirm what the value of 'intrinsic.npy' is in the robot car dataset.

Thx!!

Absolutely correct. In fact, the values in intrinsic.npy are officially provided by robotcar python-sdk. When we crop and resize the images, intrinsic is also processed in the same way. (You can easily find these in dataset/robotcar.py(nuscenes.py) *_with_intrinsic)

goxq commented 1 year ago

Thx!!