Problem in using voxblox: Mesh updating in rviz delay a lot

[JiaruiWang-Jill]

HI, I met some problem in using voxblox to generate mesh.

The mesh updating seems really wired in rviz. Sometimes it does not update for around 40s, sometimes it just stop update.

Please check the following video. https://drive.google.com/file/d/1mlzJZ7KmZeGPLfSBunnDYoTYI1SSfaGb/view?usp=sharing

Can you give more suggestion?

Thx in advance!

[helenol]

Hmm, looks like perhaps it's falling behind in processing... Can you show us your launch files and can you also upload the console output of voxblox?

[ZacharyTaylor]

The output you posted seems to be from orb slam, could you please provide the output of voxblox as it would be interesting to see the timings, any transform related messages and the number of points being integrated.

[JiaruiWang-Jill]

Sorry it is my fault. Launch file is still same. try (copy).launch.txt

When rviz shows mesh but has delay, the output is like this. ouput.txt

But just now when I try again, rviz fail to show anything. Output is as follow. output1.txt

[ZacharyTaylor]

The issue is with the transformation queue. I am a bit puzzled by whats going on though. Searching through the logs for warnings the transforms seems to span from when the node was started to the current time and the pointclouds seem to be coming in the same range. Does your pose source also publish a tf, as its possible that may not have these same issues.

Alternatively it would help us debug if you could send us a short bag that has the issue.

[JiaruiWang-Jill]

As for the publisher side, I think it also publish a tf. But in launch, I already remap <remap from="transform" to="/orb_slam_2_ros_node/transform_cam" /> Just for reference, I uploaded the publisher code. interface_rgbd (copy).txt interface (copy).txt

I record this bag file and hope this can help. https://drive.google.com/file/d/1MXOxdSvOF6z9c4-N7AN2hg6cXB9V-h_E/view?usp=sharing This one fail to show anything in rviz. I use rosbag record -a to record it. And the output is as follow. output1.txt

Plus, when I record rosbag, the terminal running roslaunch freenect_launch freenect.launch kept outputing error

[ERROR] [1535380840.727341416]: cv_bridge exception: '[32FC1] is not a color format. but [bgr8] is. The conversion does not make sense' [ERROR] [1535380840.745292403]: Cannot stream RGB and IR at the same time. Streaming RGB only. [ERROR] [1535380840.805041228]: Compressed Depth Image Transport - Compression requires single-channel 32bit-floating point or 16bit raw depth images (input format is: bayer_grbg8).

I guess it is the problem of using rosbag record -a. But I am not sure which topics are needed to debug, so I record all. If there is any need to record again, could you tell me which topics must be included?

[ZacharyTaylor]

I had a look at the rosbag, unfortunately using rosbag info revealed an issue. When you record with rosbag record -a it tries to record every topic including many that aren't usually generated unless there is a subscriber. For example it generates and records all the images 3 times with 3 different compression levels. Trying to record this much info causes it to drop most of the messages.

The only topics we need for debugging are the pointcloud and the transform topic. In your case these can be recorded via rosbag record /camera/depth_registered/points /orb_slam_2_ros_node/transform_cam

[JiaruiWang-Jill]

This time I only record point cloud and transform. https://drive.google.com/file/d/19V_AHNPz2t1ZgQxTk7EEYJSQbpdqEndH/view?usp=sharing (I deleted previous comments) Please check and thxxxxxxxxx!

[ZacharyTaylor]

I had a quick look and found the bag ran without issues on my laptop. However, I saw two possible issues. 1) The transforms generally arrive after the pointclouds and are at quite a different rate. We actually made changes to how we handle transforms last week that account for these issues. Make sure you have pulled and built the latest version of the voxblox master.

2) On my laptop the rate of pointclouds exceeded the integration rate causing it to fall behind and drop some data. To ensure I didn't hit this issue I just set <param name="min_time_between_msgs_sec" value="0.1" /> to limit incoming data to 10 Hz

[JiaruiWang-Jill]

Thank a lot. I changed voxblox into newest version and modified the launch file as you said. But my system still has large latency and sometimes it fails to show mesh in rviz. What could the problem be ?

[ZacharyTaylor]

I believe the issue isn't that there is latency in the mesh, but that voxblox is not using the measurements and so not updating the map. It might still be an issue with voxblox falling behind, orbslam tends to be a serious resource hog, what is your cpu usage like when running?

[JiaruiWang-Jill]

Thanks a lot!

It is because ORBSLAM2 use 100% CPU, voxblox fall off as a result. I changed the voxblox line to force it use 2 cores and the whole system perform well now.

[mfinean]

Thanks a lot!

It is because ORBSLAM2 use 100% CPU, voxblox fall off as a result. I changed the voxblox line to force it use 2 cores and the whole system perform well now.

What change did you make to solve this?