No streaming in Rviz when no marker is on the frame?

[EddieMG]

if I make the drone yaw I can see that the streaming on rviz stops once there are no markers visible in the frame. I tested adding more markers, and indeed if I place enough of them around the drone, the streaming continues.

Is there a reason for this?

Reproducible with: ros2 launch tello_gazebo vlam_launch.py And another terminal running the command : ros2 service call /tello_action tello_msgs/TelloAction "{cmd: 'takeoff'}" ros2 service call /tello_action tello_msgs/TelloAction "{cmd: 'rc 0 0 0 0.5'}"

[clydemcqueen]

/drone1/image_marked is published only when there are markers in view.

/drone1/image_raw should always be published.

[EddieMG]

Running ros2 topic echo /drone1/image_raw sensor_msgs/msg/Image it looks like we are receiving images since the header displays a continuous increase on the timestamp. Nevertheless, the issue appears only on Rviz, when the camera topic, both marked and raw stops when markers leave the frame.

I'm still diving into the code. If I find something useful, I'll post here.

[clydemcqueen]

Got it. I should have noticed that the issue title is "in rviz". I was able to repro.

Rviz won't draw the image if the frame_id isn't in the TF tree, or in this case, if it's in the TF tree but it's stale. fidicual_vlam only publishes the map->base_link_1 transform when it has a good localization. So when the markers are not visible no transform is published.

I solved this in a different project by providing a node (FiducialLocalizer) that subscribes to the outputs of fiducial_vlam and does a few things:

• it rejects poses where the markers are too far away. This is a simple way to prune bad poses.
• it publishes the map->base transform using a timer so there is always a good transform for Rviz, Nav2 and other systems. If there are no markers then the old transform is published. When the markers re-appear the robot appears to jump to the new location.

Code here: https://github.com/clydemcqueen/orca3/blob/main/orca_localize/src/fiducial_localizer.cpp

Description here: https://github.com/clydemcqueen/orca3/tree/main/orca_localize

tello_ros is designed to be a low-level driver. I think something like FiducialLocalizer should live in the application layer.

[clydemcqueen]

This is addressed in Foxy by https://github.com/ros2/rviz/pull/781

[EddieMG]

Makes sense. Thanks for the detailed explanation! Feel free to close the issue.