search

dtc111111 / Multi-Modal-UAV

We win the 1st place in UG2+, a task in CVPR 2024 UAV Tracking and Pose-Estimation Challenge.
MIT License
16 stars 3 forks source link

Functions of each file? #4

Open hxlei0827 opened 4 weeks ago

hxlei0827 commented 4 weeks ago

Hi, I am only focusing on the trajectory estimation part of your project, and I am trying to understand function of each file. Here are my understandings and my questions.

  1. In the pre_propcessing folder under point_cloud_processing, the "pre_processing_lidar_360.py" is used to extract the dynamic point of lidar_360 data. Question: why you do not use the same process in livox_avia data? I find that only FPS is used in "pre_processing_livox_avia.py".
  2. To train the LSTM for temporal analysis, I found you use"build_lidar_360_dataset.py" to get the feature set for training LSTM in"train_lidara_detector.py". Question: why you do not use livox_avia to build feature set for training LSTM? in other words, why there does not exist a file named "build_livox_avia_dataset.py", is it enough to use the LSTM model trained by Lidar_360 features only?
  3. I just want to make sure that the flow of running this project is： First, use "build_lidar360_dataset.py" to generate the lidar_360 feature set. Secondly, use the "train_lidara_detector.py" to train the LSTM. Thirdly, use the "pre_process_lidar_360.py", which includes LSTM model, and "pre_process_livox_avia.py" to get processed data. and Then use "pre_process_fusion.py" for fusion.

Thanks in advance, please correct me if my understanding is not correct!

hxlei0827 commented 4 weeks ago

Also, May I know what is "train_bias_correction.py" used for? what is the loaded "depth_correction_filtered.mat" in this py file.

ZHOUYI1023 commented 4 weeks ago

You can visualise the point cloud from each sensor. This dataset features a sensor suite with non-overlapping FoVs. Only the point cloud from lidar_360 contains a lot of static background, so we train a simple detector to detect the UAV from the lidar_360 point cloud. The bias correction is used to correct the potential measurement bias offset at long distances. The mat file contains the cluster centers and the corresponding ground truth.