search

ethz-asl / panoptic_mapping

A flexible submap-based framework towards spatio-temporally consistent volumetric mapping and scene understanding.
BSD 3-Clause "New" or "Revised" License
278 stars 32 forks source link

Unsynchronization and error in updateSubmap() #14

Closed SgtVincent closed 3 years ago

SgtVincent commented 3 years ago

Hi Lukas,

I tried to configure the panoptic_mapping package on my new PC since I want to use the graphics card on it for real-time computation. Yet due to the hardware-driver compatbility I can only use Ubuntu 20.04.

I successfully built the project, but when I ran the demo, I found that the the construction/publishing of mesh seems quite slow and out of sync with the motion of camera frame in RVIZ. There is an enormous delay between the movement of the camera frame and the appearance of the mesh.

Even after the end of the bag file, the mesh update process is still running ( but quite slow). Then it runs into an unknown error. I doubt if it is caused by the unsynchronization.

Could you please give some hints about the cause of this? The related log is below:

W0416 09:29:23.920747 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 3.362259445s into the past.  Requested time 1605017369.923819542 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.921140 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 3.024920696s into the past.  Requested time 1605017370.261158228 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.921512 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.678477480s into the past.  Requested time 1605017370.607601404 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.921911 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.351890960s into the past.  Requested time 1605017370.934188128 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.922278 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.014413965s into the past.  Requested time 1605017371.271665096 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.922631 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.683479630s into the past.  Requested time 1605017371.602599382 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.922986 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.360916383s into the past.  Requested time 1605017371.925162554 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.923352 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.992926696s into the past.  Requested time 1605017372.293152332 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.923722 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.669217580s into the past.  Requested time 1605017372.616861343 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.924079 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.326745558s into the past.  Requested time 1605017372.959333420 but the earliest data is at time 1605017373.286078930, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:23.924474 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.327113512s into the past.  Requested time 1605017373.286078930 but the earliest data is at time 1605017373.613192558, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:28.456105 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 71 + integration: 2965 + change: 0 + visual: 1494 = 4531ms)
W0416 09:29:28.456730 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 4.046588923s into the past.  Requested time 1605017373.944589376 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.457131 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 3.712568924s into the past.  Requested time 1605017374.278609514 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.457497 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 3.397104514s into the past.  Requested time 1605017374.594073772 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.457866 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 3.042720641s into the past.  Requested time 1605017374.948457718 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.458220 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.709790756s into the past.  Requested time 1605017375.281387568 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.458741 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.364874763s into the past.  Requested time 1605017375.626303673 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.459097 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.000442074s into the past.  Requested time 1605017375.990736246 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.459451 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.690820713s into the past.  Requested time 1605017376.300357580 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.459811 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.284152408s into the past.  Requested time 1605017376.707026005 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.460175 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.026710062s into the past.  Requested time 1605017376.964468241 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.460530 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.682723611s into the past.  Requested time 1605017377.308454752 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.460888 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.370004968s into the past.  Requested time 1605017377.621173382 but the earliest data is at time 1605017377.991178274, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:28.461237 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.127738458s into the past.  Requested time 1605017377.991178274 but the earliest data is at time 1605017378.118916750, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:31.918720 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 75 + integration: 2494 + change: 0 + visual: 887 = 3457ms)
W0416 09:29:31.919297 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.987589521s into the past.  Requested time 1605017378.601023436 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.919859 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.637597758s into the past.  Requested time 1605017378.951015234 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.920235 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.302431612s into the past.  Requested time 1605017379.286181211 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.920600 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.978180917s into the past.  Requested time 1605017379.610431910 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.920964 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.630929542s into the past.  Requested time 1605017379.957683325 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.921340 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.337194904s into the past.  Requested time 1605017380.251417875 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.921705 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.004092449s into the past.  Requested time 1605017380.584520340 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.922072 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.644855255s into the past.  Requested time 1605017380.943757534 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.922430 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.286332873s into the past.  Requested time 1605017381.302279949 but the earliest data is at time 1605017381.588612795, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:31.922794 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.034806381s into the past.  Requested time 1605017381.588612795 but the earliest data is at time 1605017381.623419285, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:35.651805 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 74 + integration: 2718 + change: 0 + visual: 935 = 3728ms)
W0416 09:29:35.652314 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.733728249s into the past.  Requested time 1605017382.259883165 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.652737 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.392772164s into the past.  Requested time 1605017382.600839138 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.653132 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.063864897s into the past.  Requested time 1605017382.929746389 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.653497 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.720581760s into the past.  Requested time 1605017383.273029566 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.653857 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.403507593s into the past.  Requested time 1605017383.590103865 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.654207 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.059471012s into the past.  Requested time 1605017383.934140444 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.654564 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.712723489s into the past.  Requested time 1605017384.280887842 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.654913 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.360051756s into the past.  Requested time 1605017384.633559704 but the earliest data is at time 1605017384.993611336, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:35.655269 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.334813311s into the past.  Requested time 1605017384.993611336 but the earliest data is at time 1605017385.328424692, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:39.665402 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 72 + integration: 2880 + change: 0 + visual: 1057 = 4009ms)
W0416 09:29:39.665992 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 3.296952642s into the past.  Requested time 1605017385.690282345 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.666409 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.972384378s into the past.  Requested time 1605017386.014850616 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.666786 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.672943477s into the past.  Requested time 1605017386.314291477 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.667150 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.346161108s into the past.  Requested time 1605017386.641073942 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.667505 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.021839026s into the past.  Requested time 1605017386.965395927 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.667871 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.670098242s into the past.  Requested time 1605017387.317136765 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.668229 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.347513787s into the past.  Requested time 1605017387.639721155 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.668593 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.997993928s into the past.  Requested time 1605017387.989241123 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.668957 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.631336907s into the past.  Requested time 1605017388.355898142 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.670078 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.260955374s into the past.  Requested time 1605017388.726279497 but the earliest data is at time 1605017388.987234831, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:39.670455 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.346477864s into the past.  Requested time 1605017388.987234831 but the earliest data is at time 1605017389.333712816, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:42.327642 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 61 + integration: 2318 + change: 0 + visual: 276 = 2656ms)
W0416 09:29:42.328330 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.359689600s into the past.  Requested time 1605017389.646519423 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.328728 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.038731819s into the past.  Requested time 1605017389.967477322 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.329116 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.618196955s into the past.  Requested time 1605017390.388012171 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.329494 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.337878469s into the past.  Requested time 1605017390.668330669 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.329874 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.981905755s into the past.  Requested time 1605017391.024303436 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.330233 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.686298739s into the past.  Requested time 1605017391.319910288 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.330596 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.348469008s into the past.  Requested time 1605017391.657740116 but the earliest data is at time 1605017392.006209135, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:42.330962 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.031402793s into the past.  Requested time 1605017392.006209135 but the earliest data is at time 1605017392.037611961, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:45.984400 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 65 + integration: 2659 + change: 0 + visual: 927 = 3653ms)
W0416 09:29:45.984999 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.659971079s into the past.  Requested time 1605017392.666702271 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.985409 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.340158105s into the past.  Requested time 1605017392.986515284 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.985951 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 2.014286095s into the past.  Requested time 1605017393.312387228 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.986346 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.676987820s into the past.  Requested time 1605017393.649685621 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.986716 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 1.324782025s into the past.  Requested time 1605017394.001891375 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.987090 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.984794860s into the past.  Requested time 1605017394.341878653 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.987493 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.706773596s into the past.  Requested time 1605017394.619899750 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.987896 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.377081740s into the past.  Requested time 1605017394.949591637 but the earliest data is at time 1605017395.326673508, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
W0416 09:29:45.988313 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.316155156s into the past.  Requested time 1605017395.326673508 but the earliest data is at time 1605017395.642828465, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
[rosbag-2] process has finished cleanly
log file: /home/junting/.ros/log/4fb047ac-9e85-11eb-a2d1-af46e70c6af9/rosbag-2*.log
I0416 09:29:52.151887 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 76 + integration: 2812 + change: 0 + visual: 3274 = 6163ms)
W0416 09:29:52.152420 27242 input_synchronizer.cpp:117] Unable to lookup transform between 'airsim_drone/Depth_cam' and 'world' (Lookup would require extrapolation 0.088299534s into the past.  Requested time 1605017395.955047369 but the earliest data is at time 1605017396.043346882, when looking up transform from frame [airsim_drone/Depth_cam] to frame [world]), skipping inputs.
I0416 09:29:58.360947 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 82 + integration: 2816 + change: 0 + visual: 3309 = 6208ms)
I0416 09:30:04.702169 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 95 + integration: 2807 + change: 0 + visual: 3437 = 6340ms)
I0416 09:30:11.064498 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 84 + integration: 2832 + change: 0 + visual: 3445 = 6361ms)
I0416 09:30:17.432153 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 83 + integration: 2949 + change: 0 + visual: 3334 = 6367ms)
I0416 09:30:23.765028 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 71 + integration: 3000 + change: 0 + visual: 3260 = 6332ms)
I0416 09:30:30.269341 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 72 + integration: 3142 + change: 0 + visual: 3288 = 6503ms)
I0416 09:30:36.738009 27242 panoptic_mapper.cpp:175] Processed input data.
(tracking: 72 + integration: 3127 + change: 0 + visual: 3268 = 6468ms)
panoptic_mapper_node: /usr/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:117: Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType Eigen::DenseCoeffsBase<Derived, 0>::operator()(Eigen::Index, Eigen::Index) const [with Derived = Eigen::Matrix<float, -1, -1>; Eigen::DenseCoeffsBase<Derived, 0>::CoeffReturnType = const float&; Eigen::Index = long int]: Assertion `row >= 0 && row < rows() && col >= 0 && col < cols()' failed.
*** Aborted at 1618558239 (unix time) try "date -d @1618558239" if you are using GNU date ***
PC: @     0x7f162370218b gsignal
*** SIGABRT (@0x3e800006a6a) received by PID 27242 (TID 0x7f16077fe700) from PID 27242; stack trace: ***
    @     0x7f1623ddc781 google::(anonymous namespace)::FailureSignalHandler()
    @     0x7f1623e0a3c0 (unknown)
    @     0x7f162370218b gsignal
    @     0x7f16236e1859 abort
    @     0x7f16236e1729 (unknown)
    @     0x7f16236f2f36 __assert_fail
    @     0x7f1622e8fe43 Eigen::DenseCoeffsBase<>::operator()()
    @     0x7f1622e8ecab panoptic_mapping::InterpolatorBilinear::interpolateDepth()
    @     0x7f1622e78ec1 panoptic_mapping::ProjectiveIntegrator::computeVoxelDistanceAndWeight()
    @     0x7f1622e78aad panoptic_mapping::ProjectiveIntegrator::updateBlock()
    @     0x7f1622e78797 panoptic_mapping::ProjectiveIntegrator::updateSubmap()
    @     0x7f1622e77f02 _ZZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS_16SubmapCollectionEPNS_9InputDataEENKUlvE_clEv
    @     0x7f1622e7bc76 _ZSt13__invoke_implIbZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS0_16SubmapCollectionEPNS0_9InputDataEEUlvE_JEET_St14__invoke_otherOT0_DpOT1_
    @     0x7f1622e7bc1e _ZSt8__invokeIZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS0_16SubmapCollectionEPNS0_9InputDataEEUlvE_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOS8_DpOS9_
    @     0x7f1622e7bbb2 _ZNSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS2_16SubmapCollectionEPNS2_9InputDataEEUlvE_EEE9_M_invokeIJLm0EEEEbSt12_Index_tupleIJXspT_EEE
    @     0x7f1622e7bb32 _ZNSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS2_16SubmapCollectionEPNS2_9InputDataEEUlvE_EEEclEv
    @     0x7f1622e7b8e8 _ZNKSt13__future_base12_Task_setterISt10unique_ptrINS_7_ResultIbEENS_12_Result_base8_DeleterEENSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNSA_16SubmapCollectionEPNSA_9InputDataEEUlvE_EEEEbEclEv
    @     0x7f1622e7b5f3 _ZNSt17_Function_handlerIFSt10unique_ptrINSt13__future_base12_Result_baseENS2_8_DeleterEEvENS1_12_Task_setterIS0_INS1_7_ResultIbEES3_ENSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNSD_16SubmapCollectionEPNSD_9InputDataEEUlvE_EEEEbEEE9_M_invokeERKSt9_Any_data
    @     0x7f1622e6e942 std::function<>::operator()()
    @     0x7f1622e6d516 std::__future_base::_State_baseV2::_M_do_set()
    @     0x7f1622e71d29 _ZSt13__invoke_implIvMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEPS1_JS9_SA_EET_St21__invoke_memfun_derefOT0_OT1_DpOT2_
    @     0x7f1622e70247 _ZSt8__invokeIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EENSt15__invoke_resultIT_JDpT0_EE4typeEOSF_DpOSG_
    @     0x7f1622e6e67a _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE_clEv
    @     0x7f1622e6e6b1 _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENKUlvE0_clEv
    @     0x7f1622e6e6c6 _ZZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_ENUlvE0_4_FUNEv
    @     0x7f1623e0747f __pthread_once_slow
    @     0x7f1622e6596a __gthread_once()
    @     0x7f1622e6e787 _ZSt9call_onceIMNSt13__future_base13_State_baseV2EFvPSt8functionIFSt10unique_ptrINS0_12_Result_baseENS4_8_DeleterEEvEEPbEJPS1_S9_SA_EEvRSt9once_flagOT_DpOT0_
    @     0x7f1622e6d2e8 std::__future_base::_State_baseV2::_M_set_result()
    @     0x7f1622e7b062 _ZZNSt13__future_base17_Async_state_implINSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS4_16SubmapCollectionEPNS4_9InputDataEEUlvE_EEEEbEC4EOSC_ENKUlvE_clEv
    @     0x7f1622e7c716 _ZSt13__invoke_implIvZNSt13__future_base17_Async_state_implINSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS5_16SubmapCollectionEPNS5_9InputDataEEUlvE_EEEEbEC4EOSD_EUlvE_JEET_St14__invoke_otherOT0_DpOT1_
    @     0x7f1622e7c6cb _ZSt8__invokeIZNSt13__future_base17_Async_state_implINSt6thread8_InvokerISt5tupleIJZN16panoptic_mapping20ProjectiveIntegrator12processInputEPNS5_16SubmapCollectionEPNS5_9InputDataEEUlvE_EEEEbEC4EOSD_EUlvE_JEENSt15__invoke_resultIT_JDpT0_EE4typeEOSI_DpOSJ_
[panoptic_mapper-3] process has died [pid 27242, exit code -6, cmd /home/junting/panoptic_ws/devel/lib/panoptic_mapping_ros/panoptic_mapper_node color_image_in:=data/color_image depth_image_in:=data/depth_image segmentation_image_in:=data/segmentation_image labels_in:=data/segmentation_labels __name:=panoptic_mapper __log:=/home/junting/.ros/log/4fb047ac-9e85-11eb-a2d1-af46e70c6af9/panoptic_mapper-3.log].
log file: /home/junting/.ros/log/4fb047ac-9e85-11eb-a2d1-af46e70c6af9/panoptic_mapper-3*.log
Schmluk commented 3 years ago

Hi @SgtVincent ,

The problem in your case is that the integration time for the measurements is extremely slow (6s at some point per image, should be max 100ms for the given configuration). This is also where the second error stems from because after some time it can no longer lookup the transforms (tf has a fixed buffer size).

Are you using the release/alpha branch? Also did you setup your catkin workspace as described, i.e. you built in release mode? Another explanation would be if you drastically reduced the voxelsize.

Let me know if this helps, Lukas

SgtVincent commented 3 years ago

Thanks Lukas!

The issue is that I set cmake build flag to Debug for gdb tracing since I want to look into the dynamic value of variables & objects. I change the flag back to Release and then it works! Do you know any workaround to enable gdb tracing while running the demo normally? Yet if there is no simple workaround, I think it is also fine for me to explore the code though.

Schmluk commented 3 years ago

Hi Junting,

You can e.g. use the mode RelWithDebInfo (which is a bit of a mixture between release and debug, maybe that helps). Also I don't think you really need to look into what panoptic_mapping does in detail, you can simply look at the submap collection after each image was integrated and see what changed.

SgtVincent commented 3 years ago

OK, thanks for your suggestion. I will try to only subscribe to the published topics and avoid modifying/looking into the panoptic_mapping.