Implementing rviz_camera_stream in rviz Window

[DieVoegel]

Hi everyone, I'm new to ROS and I'm trying to get an Image from an existing rviz/moveit window.

I can open rviz_camera_stream with the demo.launch file. But I don't know how to modify the code. I like to implement the stream in another rviz-window. I also would like to move and zoom with the camera.

Could some body help me or tell me where to find a good documentation about this Code?

Greetings

[lucasw]

You can add the camera stream to an existing rviz window by clicking on the Add button on the bottom of the rviz Displays widget, then scroll down to the rviz_camera_stream folder and select CameraPub.

The two things that need to be set up externally are a camera_info topic to define what tf frame the camera will use, and the camera properties for what resolution and angle-of-view it will have. The demo.launch files demonstrates a static transform publisher and a rostopic pub to camera info topics for this.

Being able to move the camera with the mouse can be accomplished with https://github.com/lucasw/rviz_interactive_tf .

It would be nice to be able to have the camera track the current main window rviz view- one route to doing that would be to have rviz (optionally) publish the transform of the current view onto /tf- I've wanted this feature for a while but haven't made any progress on it- ideally it could be accomplished with a plugin rather than changing the base rviz code.

Being able to zoom the camera interactively would be nice, the way to do that now would be to have live control over the fx fy parameters in the source camera info topic. https://github.com/lucasw/vimjay/blob/master/scripts/dr_camera_info.py is a somewhat slow and clunky way to do this, with a little work it could be improved. But to get to the point of tying a mousewheel or keyboard controls to zooming would require a different approach.

[DieVoegel]

Thanks for the Answer.

Can you give me a hint, how to set up externally the camera_info topic and the camera properties? I guess with

<group ns="camera1">
  <node pkg="tf" type="static_transform_publisher" name="camera_broadcaster" args="0.1 0.1 -0.5 0 0 0 1 map camera1 10"/>

  <node name="camera_info" pkg="rostopic" type="rostopic" 
    args="pub camera_info sensor_msgs/CameraInfo '{header: {seq: 0, stamp: {secs: 0, nsecs: 0}, frame_id: 'camera1'}, height: 480, width: 640, distortion_model: 'plumb_bob', D: [0], K: [500.0, 0.0, 320, 0.0, 500.0, 240.0, 0.0, 0.0, 1.0], R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0], P: [500.0, 0.0, 320, 0.0, 0.0, 500, 240, 0.0, 0.0, 0.0, 1.0, 0.0], binning_x: 0, binning_y: 0, roi: {x_offset: 0, y_offset: 0, height: 480, width: 640, do_rectify: false}}' -r 2"
    output="screen"/>
</group>

I can setup a camera Info topic. But how can I set the camera properties, when I'm using rviz with another config file? I only found people saying, you need to calibrate your camera. But this is hard with a virtual camera.

I don't need to move the camera by mouse. It's enough when I can move the camera by code e.g. with a subscriber node. Maybe fixing it at an tf-frame and moving that tf-frame would be an Idea.

[lucasw]

In your example launch file you are going to have a topic camera1/camera_info that has the input camera_info for the rviz camera stream plugin. The plugin has a field that specifies the input camera_info topic.

You don't have to calibrate your virtual camera, the input camera info K/P parameters define what the virtual camera calibration is (it just uses P, but I set K to the same values because downstream nodes may use those numbers). Mostly rviz uses the fx and fy and width and height pixels number to determine the field of view- a larger fx results in a narrower horizontal field of view, and similar for fy. If you want your virtual camera to resemble a real camera then you could copy the resolution and K/P numbers calibration of the real camera, but it won't simulate distortion.

When rviz publishes the image it will publish a new output camera_info topic that is synchronized to the output images.

[DieVoegel]

Thank you. Now I get the Image of my RViz-scene.