Visualizing data from the robotic arm's camera

[Hasan-atc]

Bug/Error: I added the Visualizations Package system you specified to unity, but I can't find a button to visualize the frames when the robotic arm is moving in the HUD window. In short, I can't watch streaming camera footage. Can you help?

Expectation: I want to visualize the images from the camera while the robotic arm is operating.

Environment :

• Unity 2020.3.0f1
• Ubuntu 20.04, ROS Noetic
• Docker

[at669]

Thanks for making this ticket! Can you clarify the steps you took to add and use the package? Specifically:

• What version of the Visualizations Package is in your project? (You can find this information in the Window > Package Manager)
• What version of the ROS-TCP-Connector is in your project?
• What version of the ROS-TCP-Endpoint are you using in your Docker container?

When you enter Play mode, what does the ROS TCP Connection HUD look like in your Game view? Do you see buttons for Topics, Transforms, or Layout (i.e. the Visualizations package is fully added, but you're not seeing the correct topic)? Do you see just the ROS IP (the ROS Connection is there, but visualizations is not)? Or do you see no HUD at all in your Game view? You may need to add or refer to an instance of your ROS TCP Connector if no HUD is appearing and no connection is being made.

[Ticket#: AIRO-1658]

[Hasan-atc]

First of all, thank you for taking the time to answer my question. :)

Visualizations Package => 0.7.0 ROS TCP Connector => 0.7.0 Docker Version => 20.10.7 ($ docker version)

Exactly what I want to do is to see the images from the camera instantly in the HUD or ROS window. I would be glad if you help.

[Hasan-atc]

I tried to get help from this link (https://github.com/Unity-Technologies/ROS-TCP-Connector/blob/main/com.unity.robotics.visualizations/Visualizations.md), but when I add the topic /testimage to the topic name at sensor_msgs/ComprossedImage in Unity, I can't see it in the HUD window. Can you help me please?

[at669]

Thanks for the info! Looks like your ROSConnection is connected as normal.

Could you clarify where you are publishing to the /testimage topic? If you have successfully registered a CompressedImage publisher in Unity, that should also show up in the Visualization HUD as an outgoing message as well.

[Hasan-atc]

Even though I type the name of the topic, I can't see it in the HUD window. Where am I doing wrong?

[at669]

It looks like you're currently using the Pick-and-Place demo project, which doesn't have an image publisher built-in. Do you have something doing image publishing in your scene, e.g. ROSConnection.GetOrCreateInstance().Publish("/testimage", new CompressedImageMsg(...)? The Visualizations Package only displays the already incoming and outgoing messages, but will not start publishing images from the scene's camera by default.

[Hasan-atc]

Currently, there is no visualizer equipment in my project except the stage camera. Could you please briefly talk about ways to add this and display it in the HUD window?

[at669]

Here's a previous thread https://github.com/Unity-Technologies/ROS-TCP-Connector/issues/223#issuecomment-949952098 on this topic, outlining the necessary steps to take a Unity Camera into a message and publishing it.

From there, once you're successfully publishing the message, you can use that message's topic in the Visualization Suite's Topic field for that message and visualize it as expected. Hope this helps.

[at669]

I'll close out this issue and continue discussion on your comment here: https://github.com/Unity-Technologies/ROS-TCP-Connector/issues/223#issuecomment-1038945087.