/depth_encoded rostopic not publishing any information when following ROS3D.js tutorial for Kinect PointCloud

[JohanJarvi]

Problem

So when following this tutorial: http://wiki.ros.org/ros3djs/Tutorials/Point%20Cloud%20Streaming%20from%20a%20Kinect

I get no output on the /depth_encoded topic at the end.

System information

I am running ROS Melodic on Ubuntu 18.04. I am using React to build the application and the following versions of the JavaScript libraries:

    "react": "17.0.1",
    "react-dom": "17.0.1",
    "ros3d": "^1.0.1",
    "roslib": "^1.1.0"

ROS Package versions:

depthcloud_encoder: 0.1.1
web_video_server: 0.2.1
rosbridge_server: 0.11.13
tf2_web_republisher: 0.3.2
depth_image_proc: 1.15.0

Differences compared to tutorial

• Running Turtlebot3 Gazebo instead of Physical Kinect.
• Using web_video_server instead of ros_web_video

Information about ROS topics and code

The major variable from the tutorial is I do not have a kinect and so I am running Turtlebot 3 Gazebo which has the following camera topics:

/camera/depth/camera_info
/camera/depth/image_raw
/camera/depth/points
/camera/parameter_descriptions
/camera/parameter_updates
/camera/rgb/camera_info
/camera/rgb/image_raw
/camera/rgb/image_raw/compressed
/camera/rgb/image_raw/compressed/parameter_descriptions
/camera/rgb/image_raw/compressed/parameter_updates
/camera/rgb/image_raw/compressedDepth
/camera/rgb/image_raw/compressedDepth/parameter_descriptions
/camera/rgb/image_raw/compressedDepth/parameter_updates
/camera/rgb/image_raw/theora
/camera/rgb/image_raw/theora/parameter_descriptions
/camera/rgb/image_raw/theora/parameter_updates

Complete React code

import React from "react";
import ROSLIB from "roslib";

export default class DepthCloud extends React.Component {
  constructor(props) {
    super(props);
  }

  componentDidMount() {
    const ROS3D = require("ros3d");
    // Create the main viewer.
    var viewer = new ROS3D.Viewer({
      divID: "viewer",
      width: 800,
      height: 600,
      antialias: true,
    });

    // Setup a client to listen to TFs.
    var tfClient = new ROSLIB.TFClient({
      ros: this.props.ros,
      angularThres: 0.01,
      transThres: 0.01,
      rate: 10.0,
      fixedFrame: "/odom",
    });

    // Setup Kinect DepthCloud stream
    var depthCloud = new ROS3D.DepthCloud({
      url:
        "http://localhost:8080/stream?topic=depthcloud_encoded.webm",
      f: 525.0,
    });
    depthCloud.startStream();

    // Create Kinect scene node
    var kinectNode = new ROS3D.SceneNode({
      frameID: "/camera_rgb_optical_frame",
      tfClient: tfClient,
      object: depthCloud,
    });
    viewer.scene.add(kinectNode);
  }

  render() {
    return (
      <div>
        <div id="viewer" />
      </div>
    );
  }
}

Recreating the issue

First I run these commands

roscore
roslaunch turtlebot3_gazebo turtlebot3_world.launch
roslaunch rosbridge_server rosbridge_websocket.launch
rosrun tf2_web_republisher tf2_web_republisher
rosrun web_video_server web_video_server

At this point I can now navigate to localhost:8080 and view both camera/depth/image_raw and camera/rgb/image_raw in the browser.

I can then proceed to run this command:

rosrun nodelet nodelet standalone depth_image_proc/convert_metric image_raw:=/camera/depth/image_raw image:=/camera/depth/image_float

Instead of this command that the tutorial suggests:

rosrun nodelet nodelet standalone depth_image_proc/convert_metric image_raw:=/camera/depth_registered/image_raw image:=/camera/depth_registered/image_float

At this point I do get another camera topic show up on localhost:8080 thanks to the web_video_server but I cannot view it as it provides the following error: [ERROR] [1618964197.352391082, 668.642000000]: cv_bridge exception: [16UC1] is not a color format. but [bgr8] is. The conversion does not make sense

I do however see data if I rostopic echo this new topic.

Finally I run this command:

rosrun depthcloud_encoder depthcloud_encoder_node _depth:=/camera/depth/image_float _rgb:=/camera/rgb/image_raw

Instead of the command the tutorial suggests:

rosrun depthcloud_encoder depthcloud_encoder_node _depth:=/camera/depth_registered/image_float _rgb:=/camera/rgb/image_rect_color

And this will then provide me with another rostopic that shows up on the web_video_server at localhost:8080 but cannot be viewed. I also get the following errors:

WARNING: no messages received and simulated time is active.
Is /clock being published?

When running rostopic echo /depthcloud_encoded.

So I believe that something is going wrong with the depthcloud encoder seeing as image_float does have data prior to the second last step. The above information should hopefully be enough to recreate and debug the issue.