Closed gheinrich closed 4 months ago
This PR adds a backbone definition and associated config file for the RADIO foundation model from:
[CVPR2024] AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One https://arxiv.org/abs/2312.06709
We measured metrics on the NAVI dataset, and re-ran the DINOv2 baselines for a fair comparison. Below are the corresponding logs files:
Depth estimation:
17042024-0541, dinov2_vitb14, 14, 2-5-8-11 , dense-cls , bindepth_dpt_k3, 10, 1.50, 0.0005, 0.0, 8, NAVI_multiview_all_reldepth, NAVI_wild_all_reldepth, 0.4187, 0.6963, 0.8490, 0.1280, 0.5987, 0.8322, 0.9159, 0.0948 17042024-0612, dinov2_vitl14, 14, 5-11-17-23, dense-cls , bindepth_dpt_k3, 10, 1.50, 0.0005, 0.0, 8, NAVI_multiview_all_reldepth, NAVI_wild_all_reldepth, 0.4537, 0.7297, 0.8654, 0.1213, 0.6302, 0.8499, 0.9242, 0.0887 17042024-0717, radio_v2 , 16, 7-15-23-31, dense , bindepth_dpt_k3, 10, 1.50, 0.0005, 0.0, 8, NAVI_multiview_all_reldepth, NAVI_wild_all_reldepth, 0.4894, 0.7631, 0.8894, 0.1117, 0.6391, 0.8580, 0.9309, 0.0859
Surface normals:
17042024-1253, radio_v2 , 16, 7-15-23-31, dense , snorm_dpt_k3_UA, 10, 1.50, 5.00e-04, 0.00e+00, 8, NAVI_multiview_all, NAVI_wild_all, 0.3457, 0.6482, 0.7627, 28.6711 17042024-1314, dinov2_vitl14 , 14, 5-11-17-23, dense-cls , snorm_dpt_k3_UA, 10, 1.50, 5.00e-04, 0.00e+00, 8, NAVI_multiview_all, NAVI_wild_all, 0.3586, 0.6529, 0.7644, 28.6790
Multi-view correspondance:
18042024-0455, dinov2_vitb14, 14, 11 , dense , 1000, 0.25, NAVI_wild_all, 35.90, 54.25, 82.17, 5.24, 32.32, 49.00, 90.00, 68.82, 51.69, 29.22 18042024-0503, dinov2_vitl14, 14, 23 , dense , 1000, 0.25, NAVI_wild_all, 39.28, 57.21, 83.50, 6.05, 35.63, 52.28, 91.60, 72.59, 54.65, 32.15 18042024-0507, radio_v2 , 16, 31 , dense , 1000, 0.25, NAVI_wild_all, 40.06, 60.08, 86.12, 4.83, 35.18, 54.65, 91.97, 75.39, 58.26, 35.54
That's awesome! Thanks @gheinrich for contributing to the code base, RADIO looks pretty interesting.
This PR adds a backbone definition and associated config file for the RADIO foundation model from:
[CVPR2024] AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One https://arxiv.org/abs/2312.06709
We measured metrics on the NAVI dataset, and re-ran the DINOv2 baselines for a fair comparison. Below are the corresponding logs files:
Depth estimation:
Surface normals:
Multi-view correspondance: