jac99 / MinkLoc3D

MinkLoc3D: Point Cloud Based Large-Scale Place Recognition
MIT License
122 stars 19 forks source link

How to process the raw dataset from Oxford RebotCar dataset ? #1

Closed QLuanWilliamed closed 3 years ago

QLuanWilliamed commented 3 years ago

Hi, I am very interested in your nice work. I would like to how to process the raw dataset download from the Oxford RebotCar dataset that has a different date compared with your dataset. If you know how to pre-process the dataset, I wish you can give me some advice. Thanks a lot!

jac99 commented 3 years ago

Hi, our method works on pre-processed RobotCar datasets, prepared by authors of PointNetVLAD paper (available at: https://github.com/mikacuy/pointnetvlad). These point clouds are constructed by accumulating multiple scans from 2D LiDAR, accumulated over 20 meter drive. Such point clouds are downsampled to 4096 points, normalized so coordinates are in -1..1 range and ground plane points are removed. Which raw LiDAR data do you want to process? Original RobotCar dataset contains both 2D and 3D LiDARs.

If you want to use raw point clouds from a LiDAR for place recognition or similar purposes, you should look into Oxford Radar RobotCar (https://oxford-robotics-institute.github.io/radar-robotcar-dataset/). It contains LiDAR scans from Velodyne HDL-32E LIDAR. They are relative large point clouds with few thousand points covering area with app. 80 meter diameter. You will not need any special pre-processing for such point clouds. My network (and underlying Minkowski Engine) operates on such 3D points clouds. You can read the point cloud as triplets of (X, Y, Z) coordinates, quantize the coordinates and pass them through the network. You'll only need to change quantization step (default setting in my code is to use very small quantization step=0.01 - as point coordinates were normalized in -1..1 range). Raw scans have point coords within -80..80 range (or similar) and you need to change quantization step accordingly (0.3 meter looks reasonably good). To get reasonable results you'll need to re-train the model (as it was trained on accumulated point clouds from 2D LIDAR, which have different characteristics. Also network architecture will need tuning/amending to get the best results.

QLuanWilliamed commented 3 years ago

Hi, thanks a lot for your reply and detailed explanation.


发件人: jac99 @.> 发送时间: 2021年3月28日 20:04 收件人: jac99/MinkLoc3D @.> 抄送: Zhaoliang Luan @.>; Author @.> 主题: Re: [jac99/MinkLoc3D] How to process the raw dataset from Oxford RebotCar dataset ? (#1)

Hi, our ethod works on pre-processed RobotCar datasets, prepared by authors of PointNetVLAD paper (available at: https://github.com/mikacuy/pointnetvladhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmikacuy%2Fpointnetvlad&data=04%7C01%7C%7C67f0e49d119e4d7630ac08d8f21c4776%7C569df091b01340e386eebd9cb9e25814%7C0%7C0%7C637525550565662436%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=E71BccpSoxjj5OExP%2BThhHGv4iGS%2FVdtVsA85OF959M%3D&reserved=0). These point clouds are constructed by accumulating multiple scans from 2D LiDAR, accumulated over 20 meter drive. Such point clouds are downsampled to 4096 points, normalized so coordinates are in -1..1 range and ground plane points are removed. Which raw LiDAR data do you want to process? Original RobotCar dataset contains both 2D and 3D LiDARs.

― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjac99%2FMinkLoc3D%2Fissues%2F1%23issuecomment-808943217&data=04%7C01%7C%7C67f0e49d119e4d7630ac08d8f21c4776%7C569df091b01340e386eebd9cb9e25814%7C0%7C0%7C637525550565672421%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JPF%2BAXolWpyyPbMybGkXG74apdfuXBSaOYiMyB2uqaQ%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPBUPYZQZZKJF7WHZDVNOFTTF6DT3ANCNFSM4Z3IEOZQ&data=04%7C01%7C%7C67f0e49d119e4d7630ac08d8f21c4776%7C569df091b01340e386eebd9cb9e25814%7C0%7C0%7C637525550565677410%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=N3MrVdWQR2zQL2%2BCzFWV1Up0mIfMHMwmmotIjX6x9%2Bo%3D&reserved=0.

QLuanWilliamed commented 3 years ago

Hi, actually I would like to try to use the camera RGB image to implement camera pose estimation with supplementary data like point cloud data. However, I don't know which LiDAR(2D or 3D) of Oxford RobotCar that I can use? and I am not clear how to process the raw data. Your response gives me substantial informative ideas, I will try it based on your advice. Thanks!


发件人: jac99 @.> 发送时间: 2021年3月28日 20:04 收件人: jac99/MinkLoc3D @.> 抄送: Zhaoliang Luan @.>; Author @.> 主题: Re: [jac99/MinkLoc3D] How to process the raw dataset from Oxford RebotCar dataset ? (#1)

Hi, our ethod works on pre-processed RobotCar datasets, prepared by authors of PointNetVLAD paper (available at: https://github.com/mikacuy/pointnetvladhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmikacuy%2Fpointnetvlad&data=04%7C01%7C%7C67f0e49d119e4d7630ac08d8f21c4776%7C569df091b01340e386eebd9cb9e25814%7C0%7C0%7C637525550565662436%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=E71BccpSoxjj5OExP%2BThhHGv4iGS%2FVdtVsA85OF959M%3D&reserved=0). These point clouds are constructed by accumulating multiple scans from 2D LiDAR, accumulated over 20 meter drive. Such point clouds are downsampled to 4096 points, normalized so coordinates are in -1..1 range and ground plane points are removed. Which raw LiDAR data do you want to process? Original RobotCar dataset contains both 2D and 3D LiDARs.

― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjac99%2FMinkLoc3D%2Fissues%2F1%23issuecomment-808943217&data=04%7C01%7C%7C67f0e49d119e4d7630ac08d8f21c4776%7C569df091b01340e386eebd9cb9e25814%7C0%7C0%7C637525550565672421%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JPF%2BAXolWpyyPbMybGkXG74apdfuXBSaOYiMyB2uqaQ%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPBUPYZQZZKJF7WHZDVNOFTTF6DT3ANCNFSM4Z3IEOZQ&data=04%7C01%7C%7C67f0e49d119e4d7630ac08d8f21c4776%7C569df091b01340e386eebd9cb9e25814%7C0%7C0%7C637525550565677410%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=N3MrVdWQR2zQL2%2BCzFWV1Up0mIfMHMwmmotIjX6x9%2Bo%3D&reserved=0.