search

puzzlepaint / surfelmeshing

Real-time surfel-based mesh reconstruction from RGB-D video.
BSD 3-Clause "New" or "Revised" License
433 stars 86 forks source link

No error, but noting in the SufelMeshing window #20

Open LuoMing2 opened 4 years ago

LuoMing2 commented 4 years ago

HI, [@puzzlepaint,] This is a very good work. Recently, I am running TUM dataset-rgbd_dataset_freiburg3_long_office_household, there is no error in the terminal, but i got nothing in the SurfelMehsing window. Can you please help me out?

This is my terminal after running dataset

` ./build_RelWithDebInfo/applications/surfel_meshing/SurfelMeshing /home/roman/sequence/TUM/rgbd_dataset_freiburg3_long_office_household/ groundtruth.txt --follow_input_camera false time file:line v| 19:24:46.830 loguru.cpp:587 INFO| arguments: ./build_RelWithDebInfo/applications/surfel_meshing/SurfelMeshing /home/roman/sequence/TUM/rgbd_dataset_freiburg3_long_office_household/ groundtruth.txt --follow_input_camera false 19:24:46.830 loguru.cpp:590 INFO| Current dir: /home/roman/sequence/surfelmeshing 19:24:46.830 loguru.cpp:592 INFO| stderr verbosity: 2 19:24:46.830 loguru.cpp:593 INFO| ----------------------------------- 19:24:46.923 main.cc:629 INFO| Read dataset with 2488 frames 19:24:47.176 main.cc:866 INFO| GPU memory usage after initialization: used = 2527.44 MiB, free = 3417.19 MiB, total = 5944.62 MiB

19:24:53.957 main.cc:1341 INFO| [frame 200] #surfels: 0, #triangles (of latest mesh): 0 19:24:53.958 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 197 000000.5818 (00.002953 +- 00.000384) [00.002385,00.006430] [CUDA frame] 197 000000.2106 (00.001069 +- 00.000110) [00.000845,00.001587] Integration 197 000000.1341 (00.000681 +- 00.000107) [00.000464,00.001233] Depth bilateral filtering 197 000000.0564 (00.000287 +- 00.000019) [00.000248,00.000333] Integration - new_surfel_creation 197 000000.0471 (00.000239 +- 00.000053) [00.000217,00.000634] Integration - measurement_blending 197 000000.0315 (00.000160 +- 00.000001) [00.000158,00.000174] Upload color image 197 000000.0296 (00.000150 +- 00.000010) [00.000145,00.000203] Upload depth image 197 000000.0214 (00.000109 +- 00.000007) [00.000099,00.000154] Depth erosion 197 000000.0078 (00.000039 +- 00.000001) [00.000038,00.000041] Depth outlier filtering 197 000000.0074 (00.000038 +- 00.000003) [00.000028,00.000045] Integration - data_association 197 000000.0071 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 197 000000.0044 (00.000022 +- 00.000001) [00.000021,00.000030] Integration - regularization 197 000000.0011 (00.000005 +- 00.000001) [00.000002,00.000011] Radius computation 197 000000.0005 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 197 000000.0005 (00.000003 +- 00.000001) [00.000002,00.000010] Surfel transfer to CPU 197 000000.0005 (00.000003 +- 00.000001) [00.000002,00.000004] Integration - surfel_merging 197 000000.0005 (00.000003 +- 00.000000) [00.000002,00.000004] Integration - neighbor_update 197 000000.0005 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 197 000000.0002 (00.000001 +- 00.000001) [00.000001,00.000020] Triangulate() 197 000000.0002 (00.000001 +- 00.000000) [00.000001,00.000003]

19:24:53.958 main.cc:1681 INFO| Target FPS of 30 for integration reached 171 times, failed 26 times 19:25:00.809 main.cc:1341 INFO| [frame 400] #surfels: 0, #triangles (of latest mesh): 0 19:25:00.809 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 397 000001.1353 (00.002860 +- 00.000358) [00.002314,00.006430] [CUDA frame] 397 000000.4261 (00.001073 +- 00.000102) [00.000845,00.001657] Integration 397 000000.2687 (00.000677 +- 00.000100) [00.000464,00.001293] Depth bilateral filtering 397 000000.1170 (00.000295 +- 00.000029) [00.000236,00.000364] Integration - new_surfel_creation 397 000000.0948 (00.000239 +- 00.000060) [00.000216,00.000752] Integration - measurement_blending 397 000000.0635 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 397 000000.0593 (00.000149 +- 00.000008) [00.000145,00.000203] Upload depth image 397 000000.0430 (00.000108 +- 00.000006) [00.000099,00.000154] Depth erosion 397 000000.0157 (00.000039 +- 00.000001) [00.000038,00.000041] Depth outlier filtering 397 000000.0149 (00.000037 +- 00.000004) [00.000026,00.000045] Integration - data_association 397 000000.0142 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 397 000000.0089 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 397 000000.0022 (00.000005 +- 00.000001) [00.000002,00.000020] Surfel transfer to CPU 397 000000.0010 (00.000003 +- 00.000001) [00.000002,00.000022] Radius computation 397 000000.0010 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 397 000000.0010 (00.000003 +- 00.000001) [00.000000,00.000010] Integration - surfel_merging 397 000000.0010 (00.000003 +- 00.000001) [00.000000,00.000004] Integration - neighbor_update 397 000000.0010 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 397 000000.0005 (00.000001 +- 00.000001) [00.000001,00.000020] Triangulate() 397 000000.0003 (00.000001 +- 00.000000) [00.000001,00.000003]

19:25:00.809 main.cc:1681 INFO| Target FPS of 30 for integration reached 348 times, failed 49 times 19:25:07.727 main.cc:1341 INFO| [frame 600] #surfels: 0, #triangles (of latest mesh): 0 19:25:07.727 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 597 000001.7038 (00.002854 +- 00.000355) [00.002314,00.006430] [CUDA frame] 597 000000.6495 (00.001088 +- 00.000109) [00.000845,00.001822] Integration 597 000000.4053 (00.000679 +- 00.000106) [00.000464,00.001335] Depth bilateral filtering 597 000000.1830 (00.000306 +- 00.000032) [00.000236,00.000375] Integration - new_surfel_creation 597 000000.1427 (00.000239 +- 00.000059) [00.000215,00.000752] Integration - measurement_blending 597 000000.0955 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 597 000000.0889 (00.000149 +- 00.000007) [00.000145,00.000203] Upload depth image 597 000000.0646 (00.000108 +- 00.000005) [00.000099,00.000154] Depth erosion 597 000000.0236 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 597 000000.0227 (00.000038 +- 00.000004) [00.000026,00.000051] Integration - data_association 597 000000.0215 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 597 000000.0134 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 597 000000.0032 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 597 000000.0016 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 597 000000.0015 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - integration 597 000000.0015 (00.000003 +- 00.000001) [00.000000,00.000010] Integration - surfel_merging 597 000000.0015 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 597 000000.0015 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 597 000000.0007 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 597 000000.0005 (00.000001 +- 00.000001) [00.000001,00.000019]

19:25:07.727 main.cc:1681 INFO| Target FPS of 30 for integration reached 497 times, failed 100 times 19:25:14.563 main.cc:1341 INFO| [frame 800] #surfels: 0, #triangles (of latest mesh): 0 19:25:14.563 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 797 000002.2726 (00.002851 +- 00.000486) [00.002275,00.012542] [CUDA frame] 797 000000.8707 (00.001092 +- 00.000109) [00.000845,00.001822] Integration 797 000000.5395 (00.000677 +- 00.000101) [00.000464,00.001335] Depth bilateral filtering 797 000000.2493 (00.000313 +- 00.000037) [00.000227,00.000400] Integration - new_surfel_creation 797 000000.1899 (00.000238 +- 00.000059) [00.000215,00.000752] Integration - measurement_blending 797 000000.1275 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 797 000000.1186 (00.000149 +- 00.000007) [00.000145,00.000203] Upload depth image 797 000000.0861 (00.000108 +- 00.000005) [00.000099,00.000154] Depth erosion 797 000000.0314 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 797 000000.0305 (00.000038 +- 00.000005) [00.000025,00.000051] Integration - data_association 797 000000.0286 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 797 000000.0178 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 797 000000.0043 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 797 000000.0021 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 797 000000.0021 (00.000003 +- 00.000001) [00.000000,00.000011] Surfel transfer to CPU 797 000000.0021 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - surfel_merging 797 000000.0020 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 797 000000.0020 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 797 000000.0010 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 797 000000.0007 (00.000001 +- 00.000001) [00.000001,00.000019]

19:25:14.563 main.cc:1681 INFO| Target FPS of 30 for integration reached 658 times, failed 139 times 19:25:21.437 main.cc:1341 INFO| [frame 1000] #surfels: 0, #triangles (of latest mesh): 0 19:25:21.438 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 997 000002.8269 (00.002835 +- 00.000458) [00.002275,00.012542] [CUDA frame] 997 000001.0871 (00.001090 +- 00.000104) [00.000845,00.001822] Integration 997 000000.6719 (00.000674 +- 00.000093) [00.000464,00.001335] Depth bilateral filtering 997 000000.3129 (00.000314 +- 00.000043) [00.000202,00.000408] Integration - new_surfel_creation 997 000000.2362 (00.000237 +- 00.000055) [00.000215,00.000752] Integration - measurement_blending 997 000000.1595 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 997 000000.1481 (00.000149 +- 00.000006) [00.000145,00.000203] Upload depth image 997 000000.1075 (00.000108 +- 00.000005) [00.000099,00.000154] Depth erosion 997 000000.0393 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 997 000000.0382 (00.000038 +- 00.000005) [00.000025,00.000052] Integration - data_association 997 000000.0359 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 997 000000.0223 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 997 000000.0055 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 997 000000.0026 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 997 000000.0026 (00.000003 +- 00.000001) [00.000000,00.000011] Surfel transfer to CPU 997 000000.0026 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - surfel_merging 997 000000.0025 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 997 000000.0025 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 997 000000.0013 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 997 000000.0009 (00.000001 +- 00.000001) [00.000001,00.000019]

19:25:21.438 main.cc:1681 INFO| Target FPS of 30 for integration reached 812 times, failed 185 times 19:25:21.489 surfel_meshing_render_w:613 INFO| maxdepth: 25 19:25:21.889 surfel_meshing_render_w:613 INFO| maxdepth: 12.5 19:25:28.205 main.cc:1341 INFO| [frame 1200] #surfels: 0, #triangles (of latest mesh): 0 19:25:28.205 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1197 000003.3627 (00.002809 +- 00.000438) [00.002146,00.012542] [CUDA frame] 1197 000001.2873 (00.001075 +- 00.000107) [00.000845,00.001822] Integration 1197 000000.8048 (00.000672 +- 00.000090) [00.000464,00.001360] Depth bilateral filtering 1197 000000.3612 (00.000302 +- 00.000051) [00.000171,00.000408] Integration - new_surfel_creation 1197 000000.2826 (00.000236 +- 00.000054) [00.000215,00.000752] Integration - measurement_blending 1197 000000.1914 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 1197 000000.1779 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1197 000000.1292 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1197 000000.0471 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1197 000000.0444 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 1197 000000.0431 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1197 000000.0267 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1197 000000.0066 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 1197 000000.0031 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1197 000000.0031 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - integration 1197 000000.0031 (00.000003 +- 00.000001) [00.000000,00.000011] Integration - surfel_merging 1197 000000.0031 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1197 000000.0030 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1197 000000.0016 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1197 000000.0011 (00.000001 +- 00.000001) [00.000001,00.000019]

19:25:28.205 main.cc:1681 INFO| Target FPS of 30 for integration reached 990 times, failed 207 times 19:25:35.097 main.cc:1341 INFO| [frame 1400] #surfels: 0, #triangles (of latest mesh): 0 19:25:35.097 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1397 000003.9315 (00.002814 +- 00.000420) [00.002146,00.012542] [CUDA frame] 1397 000001.5108 (00.001081 +- 00.000102) [00.000845,00.001822] Integration 1397 000000.9367 (00.000671 +- 00.000085) [00.000464,00.001360] Depth bilateral filtering 1397 000000.4315 (00.000309 +- 00.000052) [00.000171,00.000408] Integration - new_surfel_creation 1397 000000.3285 (00.000235 +- 00.000050) [00.000215,00.000752] Integration - measurement_blending 1397 000000.2234 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 1397 000000.2076 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1397 000000.1506 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1397 000000.0550 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1397 000000.0527 (00.000038 +- 00.000006) [00.000023,00.000052] Integration - data_association 1397 000000.0503 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1397 000000.0312 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1397 000000.0077 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 1397 000000.0036 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1397 000000.0036 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - integration 1397 000000.0036 (00.000003 +- 00.000001) [00.000000,00.000011] Integration - surfel_merging 1397 000000.0036 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1397 000000.0035 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1397 000000.0019 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1397 000000.0013 (00.000001 +- 00.000001) [00.000001,00.000019]

19:25:35.097 main.cc:1681 INFO| Target FPS of 30 for integration reached 1144 times, failed 253 times 19:25:42.001 main.cc:1341 INFO| [frame 1600] #surfels: 0, #triangles (of latest mesh): 0 19:25:42.001 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1597 000004.4955 (00.002815 +- 00.000416) [00.002146,00.012542] [CUDA frame] 1597 000001.7306 (00.001084 +- 00.000108) [00.000845,00.002844] Integration 1597 000001.0714 (00.000671 +- 00.000094) [00.000464,00.002450] Depth bilateral filtering 1597 000000.4963 (00.000311 +- 00.000050) [00.000171,00.000408] Integration - new_surfel_creation 1597 000000.3761 (00.000236 +- 00.000057) [00.000215,00.001265] Integration - measurement_blending 1597 000000.2554 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 1597 000000.2373 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1597 000000.1721 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1597 000000.0628 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1597 000000.0603 (00.000038 +- 00.000006) [00.000023,00.000052] Integration - data_association 1597 000000.0574 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1597 000000.0356 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1597 000000.0088 (00.000006 +- 00.000001) [00.000002,00.000025] Radius computation 1597 000000.0042 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1597 000000.0041 (00.000003 +- 00.000001) [00.000001,00.000022] Integration - integration 1597 000000.0041 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 1597 000000.0041 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1597 000000.0040 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1597 000000.0021 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1597 000000.0015 (00.000001 +- 00.000001) [00.000001,00.000019]

19:25:42.001 main.cc:1681 INFO| Target FPS of 30 for integration reached 1302 times, failed 295 times 19:25:48.924 main.cc:1341 INFO| [frame 1800] #surfels: 0, #triangles (of latest mesh): 0 19:25:48.924 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1797 000005.0369 (00.002803 +- 00.000411) [00.002146,00.012542] [CUDA frame] 1797 000001.9337 (00.001076 +- 00.000116) [00.000845,00.003118] Integration 1797 000001.2049 (00.000671 +- 00.000102) [00.000464,00.002734] Depth bilateral filtering 1797 000000.5465 (00.000304 +- 00.000051) [00.000171,00.000408] Integration - new_surfel_creation 1797 000000.4224 (00.000235 +- 00.000059) [00.000215,00.001265] Integration - measurement_blending 1797 000000.2874 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 1797 000000.2670 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1797 000000.1935 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1797 000000.0707 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1797 000000.0669 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 1797 000000.0646 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1797 000000.0401 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1797 000000.0099 (00.000006 +- 00.000001) [00.000002,00.000025] Radius computation 1797 000000.0047 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1797 000000.0046 (00.000003 +- 00.000001) [00.000001,00.000022] Integration - integration 1797 000000.0046 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 1797 000000.0046 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1797 000000.0045 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1797 000000.0023 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1797 000000.0017 (00.000001 +- 00.000001) [00.000001,00.000021]

19:25:48.924 main.cc:1681 INFO| Target FPS of 30 for integration reached 1442 times, failed 355 times 19:25:55.819 main.cc:1341 INFO| [frame 2000] #surfels: 0, #triangles (of latest mesh): 0 19:25:55.819 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1997 000005.5753 (00.002792 +- 00.000406) [00.002146,00.012542] [CUDA frame] 1997 000002.1371 (00.001070 +- 00.000113) [00.000845,00.003118] Integration 1997 000001.3362 (00.000669 +- 00.000097) [00.000464,00.002734] Depth bilateral filtering 1997 000000.5986 (00.000300 +- 00.000052) [00.000171,00.000408] Integration - new_surfel_creation 1997 000000.4679 (00.000234 +- 00.000057) [00.000215,00.001265] Integration - measurement_blending 1997 000000.3194 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 1997 000000.2965 (00.000148 +- 00.000007) [00.000145,00.000204] Upload depth image 1997 000000.2149 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1997 000000.0785 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1997 000000.0740 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 1997 000000.0717 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1997 000000.0446 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 1997 000000.0110 (00.000006 +- 00.000001) [00.000002,00.000025] Radius computation 1997 000000.0052 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1997 000000.0052 (00.000003 +- 00.000001) [00.000001,00.000022] Integration - integration 1997 000000.0051 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 1997 000000.0051 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1997 000000.0050 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1997 000000.0026 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 1997 000000.0018 (00.000001 +- 00.000001) [00.000001,00.000021]

19:25:55.819 main.cc:1681 INFO| Target FPS of 30 for integration reached 1590 times, failed 407 times 19:26:02.577 main.cc:1341 INFO| [frame 2200] #surfels: 0, #triangles (of latest mesh): 0 19:26:02.577 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 2197 000006.0966 (00.002775 +- 00.000399) [00.002146,00.012542] [CUDA frame] 2197 000002.3439 (00.001067 +- 00.000111) [00.000845,00.003118] Integration 2197 000001.4688 (00.000669 +- 00.000094) [00.000464,00.002734] Depth bilateral filtering 2197 000000.6529 (00.000297 +- 00.000053) [00.000142,00.000408] Integration - new_surfel_creation 2197 000000.5141 (00.000234 +- 00.000055) [00.000215,00.001265] Integration - measurement_blending 2197 000000.3513 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 2197 000000.3261 (00.000148 +- 00.000006) [00.000145,00.000204] Upload depth image 2197 000000.2364 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 2197 000000.0864 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 2197 000000.0810 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 2197 000000.0789 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 2197 000000.0490 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 2197 000000.0121 (00.000006 +- 00.000001) [00.000002,00.000025] Surfel transfer to CPU 2197 000000.0057 (00.000003 +- 00.000001) [00.000001,00.000022] Radius computation 2197 000000.0057 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 2197 000000.0056 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 2197 000000.0056 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 2197 000000.0056 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 2197 000000.0028 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 2197 000000.0020 (00.000001 +- 00.000001) [00.000001,00.000021]

19:26:02.577 main.cc:1681 INFO| Target FPS of 30 for integration reached 1776 times, failed 421 times 19:26:09.383 main.cc:1341 INFO| [frame 2400] #surfels: 0, #triangles (of latest mesh): 0 19:26:09.383 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 2397 000006.6371 (00.002769 +- 00.000392) [00.002146,00.012542] [CUDA frame] 2397 000002.5592 (00.001068 +- 00.000115) [00.000845,00.003118] Integration 2397 000001.6018 (00.000668 +- 00.000099) [00.000464,00.002734] Depth bilateral filtering 2397 000000.7145 (00.000298 +- 00.000051) [00.000142,00.000408] Integration - new_surfel_creation 2397 000000.5604 (00.000234 +- 00.000057) [00.000215,00.001266] Integration - measurement_blending 2397 000000.3833 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 2397 000000.3559 (00.000148 +- 00.000007) [00.000145,00.000204] Upload depth image 2397 000000.2580 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 2397 000000.0943 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 2397 000000.0889 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 2397 000000.0861 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 2397 000000.0535 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 2397 000000.0132 (00.000006 +- 00.000001) [00.000002,00.000025] Surfel transfer to CPU 2397 000000.0062 (00.000003 +- 00.000001) [00.000001,00.000022] Radius computation 2397 000000.0062 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - surfel_merging 2397 000000.0061 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - integration 2397 000000.0061 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - neighbor_update 2397 000000.0061 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 2397 000000.0031 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 2397 000000.0022 (00.000001 +- 00.000001) [00.000001,00.000021]

19:26:09.383 main.cc:1681 INFO| Target FPS of 30 for integration reached 1958 times, failed 439 times 19:26:12.197 main.cc:1297 INFO| Waiting for final mesh ... 19:26:12.197 main.cc:1306 INFO| Got final mesh 19:39:43.343 surfel_meshing_render_w:613 INFO| maxdepth: 6.25 19:39:44.895 surfel_meshing_render_w:613 INFO| maxdepth: 3.125 19:39:45.263 surfel_meshing_render_w:613 INFO| maxdepth: 1.5625 19:39:45.583 surfel_meshing_render_w:613 INFO| maxdepth: 0.78125 19:39:45.839 surfel_meshing_render_w:613 INFO| maxdepth: 0.390625 19:39:46.207 surfel_meshing_render_w:613 INFO| maxdepth: 0.195312 19:39:47.151 surfel_meshing_render_w:613 INFO| maxdepth: 0.0976562 19:39:47.487 surfel_meshing_render_w:613 INFO| maxdepth: 0.0488281 19:39:51.938 main.cc:1758 INFO| Timing

[Integration frame - measured on CPU] 2480 000006.8491 (00.002762 +- 00.000390) [00.002146,00.012542] [CUDA frame] 2480 000002.6455 (00.001067 +- 00.000113) [00.000845,00.003118] Integration 2480 000001.6568 (00.000668 +- 00.000098) [00.000464,00.002734] Depth bilateral filtering 2480 000000.7375 (00.000297 +- 00.000050) [00.000142,00.000408] Integration - new_surfel_creation 2480 000000.5793 (00.000234 +- 00.000056) [00.000215,00.001266] Integration - measurement_blending 2480 000000.3966 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 2480 000000.3684 (00.000149 +- 00.000007) [00.000145,00.000238] Upload depth image 2480 000000.2671 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 2480 000000.0975 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 2480 000000.0919 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 2480 000000.0890 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 2480 000000.0554 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 2480 000000.0137 (00.000006 +- 00.000001) [00.000002,00.000025] Surfel transfer to CPU 2480 000000.0064 (00.000003 +- 00.000001) [00.000001,00.000022] Radius computation 2480 000000.0064 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - surfel_merging 2480 000000.0064 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - integration 2480 000000.0063 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - neighbor_update 2480 000000.0063 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 2480 000000.0032 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 2480 000000.0023 (00.000001 +- 00.000001) [00.000001,00.000021]

19:39:51.982 loguru.cpp:467 INFO| atexit

`

And this is image Screenshot from 2020-10-09 19-25-17

puzzlepaint commented 4 years ago

It seems that no surfels get created:

19:24:53.957 main.cc:1341 INFO| [frame 200] #surfels: 0, #triangles (of latest mesh): 0

One possible reason for this is that all of the depth values in the depth images get filtered away at some point during depth preprocessing. This might for example happen if an incorrect trajectory is used. Thus, I would verify that the trajectory file matches the dataset. If the trajectory is correct, I would then try using the --debug_depth_preprocessing program argument and see whether there is some preprocessing step that removes all depth values. This might help to localize the issue.

ArthurWuzh commented 2 years ago

HI, [@puzzlepaint,] This is a very good work. Recently, I am running TUM dataset-rgbd_dataset_freiburg3_long_office_household, there is no error in the terminal, but i got nothing in the SurfelMehsing window. Can you please help me out?

This is my terminal after running dataset

` ./build_RelWithDebInfo/applications/surfel_meshing/SurfelMeshing /home/roman/sequence/TUM/rgbd_dataset_freiburg3_long_office_household/ groundtruth.txt --follow_input_camera false time file:line v| 19:24:46.830 loguru.cpp:587 INFO| arguments: ./build_RelWithDebInfo/applications/surfel_meshing/SurfelMeshing /home/roman/sequence/TUM/rgbd_dataset_freiburg3_long_office_household/ groundtruth.txt --follow_input_camera false 19:24:46.830 loguru.cpp:590 INFO| Current dir: /home/roman/sequence/surfelmeshing 19:24:46.830 loguru.cpp:592 INFO| stderr verbosity: 2 19:24:46.830 loguru.cpp:593 INFO| ----------------------------------- 19:24:46.923 main.cc:629 INFO| Read dataset with 2488 frames 19:24:47.176 main.cc:866 INFO| GPU memory usage after initialization: used = 2527.44 MiB, free = 3417.19 MiB, total = 5944.62 MiB

19:24:53.957 main.cc:1341 INFO| [frame 200] #surfels: 0, #triangles (of latest mesh): 0 19:24:53.958 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 197 000000.5818 (00.002953 +- 00.000384) [00.002385,00.006430] [CUDA frame] 197 000000.2106 (00.001069 +- 00.000110) [00.000845,00.001587] Integration 197 000000.1341 (00.000681 +- 00.000107) [00.000464,00.001233] Depth bilateral filtering 197 000000.0564 (00.000287 +- 00.000019) [00.000248,00.000333] Integration - new_surfel_creation 197 000000.0471 (00.000239 +- 00.000053) [00.000217,00.000634] Integration - measurement_blending 197 000000.0315 (00.000160 +- 00.000001) [00.000158,00.000174] Upload color image 197 000000.0296 (00.000150 +- 00.000010) [00.000145,00.000203] Upload depth image 197 000000.0214 (00.000109 +- 00.000007) [00.000099,00.000154] Depth erosion 197 000000.0078 (00.000039 +- 00.000001) [00.000038,00.000041] Depth outlier filtering 197 000000.0074 (00.000038 +- 00.000003) [00.000028,00.000045] Integration - data_association 197 000000.0071 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 197 000000.0044 (00.000022 +- 00.000001) [00.000021,00.000030] Integration - regularization 197 000000.0011 (00.000005 +- 00.000001) [00.000002,00.000011] Radius computation 197 000000.0005 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 197 000000.0005 (00.000003 +- 00.000001) [00.000002,00.000010] Surfel transfer to CPU 197 000000.0005 (00.000003 +- 00.000001) [00.000002,00.000004] Integration - surfel_merging 197 000000.0005 (00.000003 +- 00.000000) [00.000002,00.000004] Integration - neighbor_update 197 000000.0005 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 197 000000.0002 (00.000001 +- 00.000001) [00.000001,00.000020] Triangulate() 197 000000.0002 (00.000001 +- 00.000000) [00.000001,00.000003]

  • CheckRemeshing: Check existing surfels 197 000000.0000 (00.000000 +- 00.000000) [00.000000,00.000000]
  • CheckRemeshing: Remesh new surfels 197 000000.0000 (00.000000 +- 00.000000) [00.000000,00.000000]

19:24:53.958 main.cc:1681 INFO| Target FPS of 30 for integration reached 171 times, failed 26 times 19:25:00.809 main.cc:1341 INFO| [frame 400] #surfels: 0, #triangles (of latest mesh): 0 19:25:00.809 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 397 000001.1353 (00.002860 +- 00.000358) [00.002314,00.006430] [CUDA frame] 397 000000.4261 (00.001073 +- 00.000102) [00.000845,00.001657] Integration 397 000000.2687 (00.000677 +- 00.000100) [00.000464,00.001293] Depth bilateral filtering 397 000000.1170 (00.000295 +- 00.000029) [00.000236,00.000364] Integration - new_surfel_creation 397 000000.0948 (00.000239 +- 00.000060) [00.000216,00.000752] Integration - measurement_blending 397 000000.0635 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 397 000000.0593 (00.000149 +- 00.000008) [00.000145,00.000203] Upload depth image 397 000000.0430 (00.000108 +- 00.000006) [00.000099,00.000154] Depth erosion 397 000000.0157 (00.000039 +- 00.000001) [00.000038,00.000041] Depth outlier filtering 397 000000.0149 (00.000037 +- 00.000004) [00.000026,00.000045] Integration - data_association 397 000000.0142 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 397 000000.0089 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 397 000000.0022 (00.000005 +- 00.000001) [00.000002,00.000020] Surfel transfer to CPU 397 000000.0010 (00.000003 +- 00.000001) [00.000002,00.000022] Radius computation 397 000000.0010 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 397 000000.0010 (00.000003 +- 00.000001) [00.000000,00.000010] Integration - surfel_merging 397 000000.0010 (00.000003 +- 00.000001) [00.000000,00.000004] Integration - neighbor_update 397 000000.0010 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 397 000000.0005 (00.000001 +- 00.000001) [00.000001,00.000020] Triangulate() 397 000000.0003 (00.000001 +- 00.000000) [00.000001,00.000003]

  • CheckRemeshing: Check existing surfels 397 000000.0000 (00.000000 +- 00.000000) [00.000000,00.000000]
  • CheckRemeshing: Remesh new surfels 397 000000.0000 (00.000000 +- 00.000000) [00.000000,00.000000]

19:25:00.809 main.cc:1681 INFO| Target FPS of 30 for integration reached 348 times, failed 49 times 19:25:07.727 main.cc:1341 INFO| [frame 600] #surfels: 0, #triangles (of latest mesh): 0 19:25:07.727 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 597 000001.7038 (00.002854 +- 00.000355) [00.002314,00.006430] [CUDA frame] 597 000000.6495 (00.001088 +- 00.000109) [00.000845,00.001822] Integration 597 000000.4053 (00.000679 +- 00.000106) [00.000464,00.001335] Depth bilateral filtering 597 000000.1830 (00.000306 +- 00.000032) [00.000236,00.000375] Integration - new_surfel_creation 597 000000.1427 (00.000239 +- 00.000059) [00.000215,00.000752] Integration - measurement_blending 597 000000.0955 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 597 000000.0889 (00.000149 +- 00.000007) [00.000145,00.000203] Upload depth image 597 000000.0646 (00.000108 +- 00.000005) [00.000099,00.000154] Depth erosion 597 000000.0236 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 597 000000.0227 (00.000038 +- 00.000004) [00.000026,00.000051] Integration - data_association 597 000000.0215 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 597 000000.0134 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 597 000000.0032 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 597 000000.0016 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 597 000000.0015 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - integration 597 000000.0015 (00.000003 +- 00.000001) [00.000000,00.000010] Integration - surfel_merging 597 000000.0015 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 597 000000.0015 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 597 000000.0007 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 597 000000.0005 (00.000001 +- 00.000001) [00.000001,00.000019]

  • CheckRemeshing: Check existing surfels 597 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000000]
  • CheckRemeshing: Remesh new surfels 597 000000.0000 (00.000000 +- 00.000000) [00.000000,00.000000]

19:25:07.727 main.cc:1681 INFO| Target FPS of 30 for integration reached 497 times, failed 100 times 19:25:14.563 main.cc:1341 INFO| [frame 800] #surfels: 0, #triangles (of latest mesh): 0 19:25:14.563 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 797 000002.2726 (00.002851 +- 00.000486) [00.002275,00.012542] [CUDA frame] 797 000000.8707 (00.001092 +- 00.000109) [00.000845,00.001822] Integration 797 000000.5395 (00.000677 +- 00.000101) [00.000464,00.001335] Depth bilateral filtering 797 000000.2493 (00.000313 +- 00.000037) [00.000227,00.000400] Integration - new_surfel_creation 797 000000.1899 (00.000238 +- 00.000059) [00.000215,00.000752] Integration - measurement_blending 797 000000.1275 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 797 000000.1186 (00.000149 +- 00.000007) [00.000145,00.000203] Upload depth image 797 000000.0861 (00.000108 +- 00.000005) [00.000099,00.000154] Depth erosion 797 000000.0314 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 797 000000.0305 (00.000038 +- 00.000005) [00.000025,00.000051] Integration - data_association 797 000000.0286 (00.000036 +- 00.000002) [00.000033,00.000058] Normal computation 797 000000.0178 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 797 000000.0043 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 797 000000.0021 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 797 000000.0021 (00.000003 +- 00.000001) [00.000000,00.000011] Surfel transfer to CPU 797 000000.0021 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - surfel_merging 797 000000.0020 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 797 000000.0020 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 797 000000.0010 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 797 000000.0007 (00.000001 +- 00.000001) [00.000001,00.000019]

  • CheckRemeshing: Check existing surfels 797 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000001]
  • CheckRemeshing: Remesh new surfels 797 000000.0000 (00.000000 +- 00.000000) [00.000000,00.000000]

19:25:14.563 main.cc:1681 INFO| Target FPS of 30 for integration reached 658 times, failed 139 times 19:25:21.437 main.cc:1341 INFO| [frame 1000] #surfels: 0, #triangles (of latest mesh): 0 19:25:21.438 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 997 000002.8269 (00.002835 +- 00.000458) [00.002275,00.012542] [CUDA frame] 997 000001.0871 (00.001090 +- 00.000104) [00.000845,00.001822] Integration 997 000000.6719 (00.000674 +- 00.000093) [00.000464,00.001335] Depth bilateral filtering 997 000000.3129 (00.000314 +- 00.000043) [00.000202,00.000408] Integration - new_surfel_creation 997 000000.2362 (00.000237 +- 00.000055) [00.000215,00.000752] Integration - measurement_blending 997 000000.1595 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 997 000000.1481 (00.000149 +- 00.000006) [00.000145,00.000203] Upload depth image 997 000000.1075 (00.000108 +- 00.000005) [00.000099,00.000154] Depth erosion 997 000000.0393 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 997 000000.0382 (00.000038 +- 00.000005) [00.000025,00.000052] Integration - data_association 997 000000.0359 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 997 000000.0223 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 997 000000.0055 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 997 000000.0026 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 997 000000.0026 (00.000003 +- 00.000001) [00.000000,00.000011] Surfel transfer to CPU 997 000000.0026 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - surfel_merging 997 000000.0025 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 997 000000.0025 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 997 000000.0013 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 997 000000.0009 (00.000001 +- 00.000001) [00.000001,00.000019]

  • CheckRemeshing: Check existing surfels 997 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000001]
  • CheckRemeshing: Remesh new surfels 997 000000.0001 (00.000000 +- 00.000001) [00.000000,00.000018]

19:25:21.438 main.cc:1681 INFO| Target FPS of 30 for integration reached 812 times, failed 185 times 19:25:21.489 surfel_meshing_render_w:613 INFO| maxdepth: 25 19:25:21.889 surfel_meshing_render_w:613 INFO| maxdepth: 12.5 19:25:28.205 main.cc:1341 INFO| [frame 1200] #surfels: 0, #triangles (of latest mesh): 0 19:25:28.205 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1197 000003.3627 (00.002809 +- 00.000438) [00.002146,00.012542] [CUDA frame] 1197 000001.2873 (00.001075 +- 00.000107) [00.000845,00.001822] Integration 1197 000000.8048 (00.000672 +- 00.000090) [00.000464,00.001360] Depth bilateral filtering 1197 000000.3612 (00.000302 +- 00.000051) [00.000171,00.000408] Integration - new_surfel_creation 1197 000000.2826 (00.000236 +- 00.000054) [00.000215,00.000752] Integration - measurement_blending 1197 000000.1914 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 1197 000000.1779 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1197 000000.1292 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1197 000000.0471 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1197 000000.0444 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 1197 000000.0431 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1197 000000.0267 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1197 000000.0066 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 1197 000000.0031 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1197 000000.0031 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - integration 1197 000000.0031 (00.000003 +- 00.000001) [00.000000,00.000011] Integration - surfel_merging 1197 000000.0031 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1197 000000.0030 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1197 000000.0016 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1197 000000.0011 (00.000001 +- 00.000001) [00.000001,00.000019]

  • CheckRemeshing: Check existing surfels 1197 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000001]
  • CheckRemeshing: Remesh new surfels 1197 000000.0001 (00.000000 +- 00.000001) [00.000000,00.000018]

19:25:28.205 main.cc:1681 INFO| Target FPS of 30 for integration reached 990 times, failed 207 times 19:25:35.097 main.cc:1341 INFO| [frame 1400] #surfels: 0, #triangles (of latest mesh): 0 19:25:35.097 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1397 000003.9315 (00.002814 +- 00.000420) [00.002146,00.012542] [CUDA frame] 1397 000001.5108 (00.001081 +- 00.000102) [00.000845,00.001822] Integration 1397 000000.9367 (00.000671 +- 00.000085) [00.000464,00.001360] Depth bilateral filtering 1397 000000.4315 (00.000309 +- 00.000052) [00.000171,00.000408] Integration - new_surfel_creation 1397 000000.3285 (00.000235 +- 00.000050) [00.000215,00.000752] Integration - measurement_blending 1397 000000.2234 (00.000160 +- 00.000001) [00.000158,00.000178] Upload color image 1397 000000.2076 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1397 000000.1506 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1397 000000.0550 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1397 000000.0527 (00.000038 +- 00.000006) [00.000023,00.000052] Integration - data_association 1397 000000.0503 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1397 000000.0312 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1397 000000.0077 (00.000005 +- 00.000001) [00.000002,00.000020] Radius computation 1397 000000.0036 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1397 000000.0036 (00.000003 +- 00.000001) [00.000002,00.000022] Integration - integration 1397 000000.0036 (00.000003 +- 00.000001) [00.000000,00.000011] Integration - surfel_merging 1397 000000.0036 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1397 000000.0035 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1397 000000.0019 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1397 000000.0013 (00.000001 +- 00.000001) [00.000001,00.000019]

  • CheckRemeshing: Check existing surfels 1397 000000.0002 (00.000000 +- 00.000000) [00.000000,00.000014]
  • CheckRemeshing: Remesh new surfels 1397 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000018]

19:25:35.097 main.cc:1681 INFO| Target FPS of 30 for integration reached 1144 times, failed 253 times 19:25:42.001 main.cc:1341 INFO| [frame 1600] #surfels: 0, #triangles (of latest mesh): 0 19:25:42.001 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1597 000004.4955 (00.002815 +- 00.000416) [00.002146,00.012542] [CUDA frame] 1597 000001.7306 (00.001084 +- 00.000108) [00.000845,00.002844] Integration 1597 000001.0714 (00.000671 +- 00.000094) [00.000464,00.002450] Depth bilateral filtering 1597 000000.4963 (00.000311 +- 00.000050) [00.000171,00.000408] Integration - new_surfel_creation 1597 000000.3761 (00.000236 +- 00.000057) [00.000215,00.001265] Integration - measurement_blending 1597 000000.2554 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 1597 000000.2373 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1597 000000.1721 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1597 000000.0628 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1597 000000.0603 (00.000038 +- 00.000006) [00.000023,00.000052] Integration - data_association 1597 000000.0574 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1597 000000.0356 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1597 000000.0088 (00.000006 +- 00.000001) [00.000002,00.000025] Radius computation 1597 000000.0042 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1597 000000.0041 (00.000003 +- 00.000001) [00.000001,00.000022] Integration - integration 1597 000000.0041 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 1597 000000.0041 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1597 000000.0040 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1597 000000.0021 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1597 000000.0015 (00.000001 +- 00.000001) [00.000001,00.000019]

  • CheckRemeshing: Check existing surfels 1597 000000.0002 (00.000000 +- 00.000000) [00.000000,00.000014]
  • CheckRemeshing: Remesh new surfels 1597 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000018]

19:25:42.001 main.cc:1681 INFO| Target FPS of 30 for integration reached 1302 times, failed 295 times 19:25:48.924 main.cc:1341 INFO| [frame 1800] #surfels: 0, #triangles (of latest mesh): 0 19:25:48.924 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1797 000005.0369 (00.002803 +- 00.000411) [00.002146,00.012542] [CUDA frame] 1797 000001.9337 (00.001076 +- 00.000116) [00.000845,00.003118] Integration 1797 000001.2049 (00.000671 +- 00.000102) [00.000464,00.002734] Depth bilateral filtering 1797 000000.5465 (00.000304 +- 00.000051) [00.000171,00.000408] Integration - new_surfel_creation 1797 000000.4224 (00.000235 +- 00.000059) [00.000215,00.001265] Integration - measurement_blending 1797 000000.2874 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 1797 000000.2670 (00.000149 +- 00.000007) [00.000145,00.000204] Upload depth image 1797 000000.1935 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1797 000000.0707 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1797 000000.0669 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 1797 000000.0646 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1797 000000.0401 (00.000022 +- 00.000001) [00.000021,00.000035] Integration - regularization 1797 000000.0099 (00.000006 +- 00.000001) [00.000002,00.000025] Radius computation 1797 000000.0047 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1797 000000.0046 (00.000003 +- 00.000001) [00.000001,00.000022] Integration - integration 1797 000000.0046 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 1797 000000.0046 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1797 000000.0045 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1797 000000.0023 (00.000001 +- 00.000002) [00.000001,00.000021] Triangulate() 1797 000000.0017 (00.000001 +- 00.000001) [00.000001,00.000021]

  • CheckRemeshing: Check existing surfels 1797 000000.0002 (00.000000 +- 00.000000) [00.000000,00.000014]
  • CheckRemeshing: Remesh new surfels 1797 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000018]

19:25:48.924 main.cc:1681 INFO| Target FPS of 30 for integration reached 1442 times, failed 355 times 19:25:55.819 main.cc:1341 INFO| [frame 2000] #surfels: 0, #triangles (of latest mesh): 0 19:25:55.819 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 1997 000005.5753 (00.002792 +- 00.000406) [00.002146,00.012542] [CUDA frame] 1997 000002.1371 (00.001070 +- 00.000113) [00.000845,00.003118] Integration 1997 000001.3362 (00.000669 +- 00.000097) [00.000464,00.002734] Depth bilateral filtering 1997 000000.5986 (00.000300 +- 00.000052) [00.000171,00.000408] Integration - new_surfel_creation 1997 000000.4679 (00.000234 +- 00.000057) [00.000215,00.001265] Integration - measurement_blending 1997 000000.3194 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 1997 000000.2965 (00.000148 +- 00.000007) [00.000145,00.000204] Upload depth image 1997 000000.2149 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 1997 000000.0785 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 1997 000000.0740 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 1997 000000.0717 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 1997 000000.0446 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 1997 000000.0110 (00.000006 +- 00.000001) [00.000002,00.000025] Radius computation 1997 000000.0052 (00.000003 +- 00.000001) [00.000002,00.000016] Surfel transfer to CPU 1997 000000.0052 (00.000003 +- 00.000001) [00.000001,00.000022] Integration - integration 1997 000000.0051 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 1997 000000.0051 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 1997 000000.0050 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 1997 000000.0026 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 1997 000000.0018 (00.000001 +- 00.000001) [00.000001,00.000021]

  • CheckRemeshing: Check existing surfels 1997 000000.0003 (00.000000 +- 00.000001) [00.000000,00.000026]
  • CheckRemeshing: Remesh new surfels 1997 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000018]

19:25:55.819 main.cc:1681 INFO| Target FPS of 30 for integration reached 1590 times, failed 407 times 19:26:02.577 main.cc:1341 INFO| [frame 2200] #surfels: 0, #triangles (of latest mesh): 0 19:26:02.577 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 2197 000006.0966 (00.002775 +- 00.000399) [00.002146,00.012542] [CUDA frame] 2197 000002.3439 (00.001067 +- 00.000111) [00.000845,00.003118] Integration 2197 000001.4688 (00.000669 +- 00.000094) [00.000464,00.002734] Depth bilateral filtering 2197 000000.6529 (00.000297 +- 00.000053) [00.000142,00.000408] Integration - new_surfel_creation 2197 000000.5141 (00.000234 +- 00.000055) [00.000215,00.001265] Integration - measurement_blending 2197 000000.3513 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 2197 000000.3261 (00.000148 +- 00.000006) [00.000145,00.000204] Upload depth image 2197 000000.2364 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 2197 000000.0864 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 2197 000000.0810 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 2197 000000.0789 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 2197 000000.0490 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 2197 000000.0121 (00.000006 +- 00.000001) [00.000002,00.000025] Surfel transfer to CPU 2197 000000.0057 (00.000003 +- 00.000001) [00.000001,00.000022] Radius computation 2197 000000.0057 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - integration 2197 000000.0056 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - surfel_merging 2197 000000.0056 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - neighbor_update 2197 000000.0056 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 2197 000000.0028 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 2197 000000.0020 (00.000001 +- 00.000001) [00.000001,00.000021]

  • CheckRemeshing: Check existing surfels 2197 000000.0003 (00.000000 +- 00.000001) [00.000000,00.000026]
  • CheckRemeshing: Remesh new surfels 2197 000000.0001 (00.000000 +- 00.000000) [00.000000,00.000018]

19:26:02.577 main.cc:1681 INFO| Target FPS of 30 for integration reached 1776 times, failed 421 times 19:26:09.383 main.cc:1341 INFO| [frame 2400] #surfels: 0, #triangles (of latest mesh): 0 19:26:09.383 main.cc:1528 INFO| Timing

[Integration frame - measured on CPU] 2397 000006.6371 (00.002769 +- 00.000392) [00.002146,00.012542] [CUDA frame] 2397 000002.5592 (00.001068 +- 00.000115) [00.000845,00.003118] Integration 2397 000001.6018 (00.000668 +- 00.000099) [00.000464,00.002734] Depth bilateral filtering 2397 000000.7145 (00.000298 +- 00.000051) [00.000142,00.000408] Integration - new_surfel_creation 2397 000000.5604 (00.000234 +- 00.000057) [00.000215,00.001266] Integration - measurement_blending 2397 000000.3833 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 2397 000000.3559 (00.000148 +- 00.000007) [00.000145,00.000204] Upload depth image 2397 000000.2580 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 2397 000000.0943 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 2397 000000.0889 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 2397 000000.0861 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 2397 000000.0535 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 2397 000000.0132 (00.000006 +- 00.000001) [00.000002,00.000025] Surfel transfer to CPU 2397 000000.0062 (00.000003 +- 00.000001) [00.000001,00.000022] Radius computation 2397 000000.0062 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - surfel_merging 2397 000000.0061 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - integration 2397 000000.0061 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - neighbor_update 2397 000000.0061 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 2397 000000.0031 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 2397 000000.0022 (00.000001 +- 00.000001) [00.000001,00.000021]

  • CheckRemeshing: Check existing surfels 2397 000000.0003 (00.000000 +- 00.000001) [00.000000,00.000026]
  • CheckRemeshing: Remesh new surfels 2397 000000.0002 (00.000000 +- 00.000000) [00.000000,00.000018]

19:26:09.383 main.cc:1681 INFO| Target FPS of 30 for integration reached 1958 times, failed 439 times 19:26:12.197 main.cc:1297 INFO| Waiting for final mesh ... 19:26:12.197 main.cc:1306 INFO| Got final mesh 19:39:43.343 surfel_meshing_render_w:613 INFO| maxdepth: 6.25 19:39:44.895 surfel_meshing_render_w:613 INFO| maxdepth: 3.125 19:39:45.263 surfel_meshing_render_w:613 INFO| maxdepth: 1.5625 19:39:45.583 surfel_meshing_render_w:613 INFO| maxdepth: 0.78125 19:39:45.839 surfel_meshing_render_w:613 INFO| maxdepth: 0.390625 19:39:46.207 surfel_meshing_render_w:613 INFO| maxdepth: 0.195312 19:39:47.151 surfel_meshing_render_w:613 INFO| maxdepth: 0.0976562 19:39:47.487 surfel_meshing_render_w:613 INFO| maxdepth: 0.0488281 19:39:51.938 main.cc:1758 INFO| Timing

[Integration frame - measured on CPU] 2480 000006.8491 (00.002762 +- 00.000390) [00.002146,00.012542] [CUDA frame] 2480 000002.6455 (00.001067 +- 00.000113) [00.000845,00.003118] Integration 2480 000001.6568 (00.000668 +- 00.000098) [00.000464,00.002734] Depth bilateral filtering 2480 000000.7375 (00.000297 +- 00.000050) [00.000142,00.000408] Integration - new_surfel_creation 2480 000000.5793 (00.000234 +- 00.000056) [00.000215,00.001266] Integration - measurement_blending 2480 000000.3966 (00.000160 +- 00.000001) [00.000158,00.000188] Upload color image 2480 000000.3684 (00.000149 +- 00.000007) [00.000145,00.000238] Upload depth image 2480 000000.2671 (00.000108 +- 00.000005) [00.000099,00.000181] Depth erosion 2480 000000.0975 (00.000039 +- 00.000001) [00.000038,00.000053] Depth outlier filtering 2480 000000.0919 (00.000037 +- 00.000006) [00.000023,00.000052] Integration - data_association 2480 000000.0890 (00.000036 +- 00.000002) [00.000033,00.000068] Normal computation 2480 000000.0554 (00.000022 +- 00.000001) [00.000021,00.000037] Integration - regularization 2480 000000.0137 (00.000006 +- 00.000001) [00.000002,00.000025] Surfel transfer to CPU 2480 000000.0064 (00.000003 +- 00.000001) [00.000001,00.000022] Radius computation 2480 000000.0064 (00.000003 +- 00.000001) [00.000002,00.000016] Integration - surfel_merging 2480 000000.0064 (00.000003 +- 00.000000) [00.000000,00.000004] Integration - integration 2480 000000.0063 (00.000003 +- 00.000001) [00.000000,00.000012] Integration - neighbor_update 2480 000000.0063 (00.000003 +- 00.000000) [00.000002,00.000004] CheckRemeshing() 2480 000000.0032 (00.000001 +- 00.000002) [00.000001,00.000027] Triangulate() 2480 000000.0023 (00.000001 +- 00.000001) [00.000001,00.000021]

  • CheckRemeshing: Check existing surfels 2480 000000.0003 (00.000000 +- 00.000001) [00.000000,00.000026]
  • CheckRemeshing: Remesh new surfels 2480 000000.0002 (00.000000 +- 00.000000) [00.000000,00.000018]

19:39:51.982 loguru.cpp:467 INFO| atexit

`

And this is image Screenshot from 2020-10-09 19-25-17

Has this problem been solved?

ArthurWuzh commented 2 years ago

It seems that no surfels get created:

19:24:53.957 main.cc:1341 INFO| [frame 200] #surfels: 0, #triangles (of latest mesh): 0

One possible reason for this is that all of the depth values in the depth images get filtered away at some point during depth preprocessing. This might for example happen if an incorrect trajectory is used. Thus, I would verify that the trajectory file matches the dataset. If the trajectory is correct, I would then try using the --debug_depth_preprocessing program argument and see whether there is some preprocessing step that removes all depth values. This might help to localize the issue.

How to use --debug_depth_preprocessing to solve the problem?

dumyCq commented 1 year ago

Just put --debug_depth_preprocessing at the end for example ./build_RelWithDebInfo/applications/surfel_meshing/SurfelMeshing ./rgbd_dataset_freiburg1_xyz groundtruth.txt --follow_input_camera false --debug_depth_preprocessing. It will pop up 5 windows related to the depth map.