ICRA Evaluation Global Localization in Meshes

[drehermarc]

Hi, this is the PR (will not be merged into master) corresponding to the ICRA evaluation. For each evaluation (whose results can be found here), I will create a commit with all unchanged files and include the used dataset.
The commit add54a7 corresponds to the evaluation set up for the data set "smb_longtrajectory..." (garage) with prrus using RHT with ball accumulator and ICP for refinement (here). The file "test_results.txt" includes the "raw data evaluation". Each line represent one test iteration (resp. frame) and is build up in the following way: gt_x gt_y gt_z gt_qw gt_qx gt_qy gt_qz x y z qw qx qy qz translation_error rotation_error time_needed prrus_error. If PRRUS failed, prrus_error will be set to -1.

[drehermarc]

Commit 05f0490 includes the corrected ground truth data of the arche_13_30_10 data set. Additionally, I made some changes to the plane extraction which should reduce the effect of smaller (noisy) objects. The new results can be found here (rht ball: garage_long_trajectory and arche_13_30_10). Commit b3088df includes the evaluation with an added plane in the map corresponding to the one caused by reflection (will be removed in the next evaluations) (results).

[drehermarc]

Commit 8273e6b includes a more accurate representation of the arche map. Both data sets (arche_18_22_12 and arche_13_30_10) have been evaluated (without correction of the offset robot-lidar yet and any added planes not appearing in the provided map).

[ethzasl-jenkins]

Test FAILed.

[ethzasl-jenkins]

Test FAILed.

[ethzasl-jenkins]

Test FAILed.

[ethzasl-jenkins]

Test FAILed.

[ethzasl-jenkins]

Test FAILed.

[drehermarc]

Commit eb96d7c includes the evaluation of different pipelines, matching a transformed dense sampled point cloud of the CAD "construct" to exactly the same point cloud. The results and visualization of the "ratio plots" (logarithmic scale) can be found here.. The variables obtained during the evaluation (translation errors, rotation errors, required time, etc.) are contained in the .mat files in the appropriate folder (the matlab script "createStatistic" used to calculate these variables can be found here).

[ethzasl-jenkins]

Test FAILed.

[drehermarc]

Commit 22621c0 includes the evaluation of the real data recorded in the arche map. On one occasion the entire CAD model was used as a map (LiDAR only in the upper right-hand room) (results) and on another occasion the localisation was carried out in the upper right-hand room only (results).

[drehermarc]

Commit e55047d includes the evaluation of PRRUS with RHT with ICP on the simulated data in the upper right-hand room of the map arche (results).

[michaelpantic]

@dreherm-student-ethz @hermannsblum I noticed our repo is suddenly around 2 GB to clone - and it seems that most of this data comes from your dataset. Please see that this gets cleaned out of the history again (deleting them and commiting might not be good enough) - as it does not belong to the source code and makes checking it out a huge pain. Thanks!

[hermannsblum]

@dreherm-student-ethz fyi we have git-lfs for all asl repositories, please use that if you add data files

[drehermarc]

@dreherm-student-ethz @hermannsblum I noticed our repo is suddenly around 2 GB to clone - and it seems that most of this data comes from your dataset. Please see that this gets cleaned out of the history again (deleting them and commiting might not be good enough) - as it does not belong to the source code and makes checking it out a huge pain. Thanks!

Hello I have removed the data sets from history on both the icra_evaluation-feature and my post-thesis branch. These should now take up significantly less memory. The closed PRs for these branches will be deleted in due time.
Thanks for the tip with git-lfs. I will take a closer look at this tomorrow.