Broken meshes for actors with the segmentation camera

[adlarkin]

Environment

• OS Version: Ubuntu 18.04
• Source or binary build? Binary, 6.0.0~pre1
• If this is a GUI or sensor rendering bug, describe your GPU and rendering system. Otherwise delete this section.
  • Rendering plugin: ogre2
  • Generally, mention all circumstances that might affect rendering capabilities:
    • running in Docker/Singularity

Description

• Expected behavior: When running the segmentation camera with actors, the actor should appear uniform
• Actual behavior: Some of the actor's body parts are detached/floating: https://github.com/ignitionrobotics/ign-gazebo/pull/853#issuecomment-922090868

Steps to reproduce

1. Add a segmentation camera and label to the demo_actor in examples/worlds/actor.sdf:
   actor.sdf diff

diff --git a/examples/worlds/actor.sdf b/examples/worlds/actor.sdf
index 346c511b..9ad0322a 100644
--- a/examples/worlds/actor.sdf
+++ b/examples/worlds/actor.sdf
@@ -151,6 +151,9 @@
       <!-- override actor's pose, which has 1m in Z -->
       <pose>0 0 0 0 0 0</pose>
       <uri>https://fuel.ignitionrobotics.org/1.0/Mingfei/models/actor</uri>
+      <plugin filename="ignition-gazebo-label-system" name="ignition::gazebo::systems::Label">
+        <label>40</label>
+      </plugin>
     </include>

     <actor name="actor_talking">
@@ -248,5 +251,56 @@
       </link>
     </model>

+    <!-- Semantic Segmentation Camera Sensor -->
+    <model name="semantic_camera">
+      <pose>-2.5 0 2.0 0 0.0 0</pose>
+      <link name="link">
+        <pose>0 0 0 0 0 0</pose>
+        <inertial>
+          <mass>0.5</mass>
+          <inertia>
+            <ixx>0.000166667</ixx>
+            <iyy>0.000166667</iyy>
+            <izz>0.000166667</izz>
+          </inertia>
+        </inertial>
+        <collision name="collision">
+          <geometry>
+            <box>
+              <size>0.1 0.1 0.1</size>
+            </box>
+          </geometry>
+        </collision>
+        <visual name="visual">
+          <geometry>
+            <box>
+              <size>0.1 0.1 0.1</size>
+            </box>
+          </geometry>
+        </visual>
+        <sensor name="semantic_segmentation_camera" type="segmentation">
+          <topic>semantic</topic>
+          <camera>
+            <segmentation_type>semantic</segmentation_type>
+            <horizontal_fov>1.57</horizontal_fov>
+            <image>
+              <width>800</width>
+              <height>600</height>
+            </image>
+            <clip>
+              <near>0.1</near>
+              <far>100</far>
+            </clip>
+          </camera>
+          <always_on>1</always_on>
+          <update_rate>30</update_rate>
+          <visualize>true</visualize>
+        </sensor>
+      </link>
+    </model>
+
   </world>
 </sdf>

2. Run the actor.sdf world: ign gazebo actor.sdf
3. Add an image display to the GUI and then set the topic of the image display to /semantic/colored_map
4. Start simulation and observe the incorrect segmentation output

Output

See https://github.com/ignitionrobotics/ign-gazebo/pull/853#issuecomment-922090868

[vavisc]

I have just encountered this problem. Here is an image of the coloured_map. Is there any progress in solving it?

[chapulina]

Hi @vavisc , no one is currently working on it, and we appreciate pull requests. Thanks!

[pw151]

I encountered the same problem when using a gpu_lidar sensor. The point cloud visualization in rviz also shows a broken mesh.

[iche033]

this issue may be fixed in garden (gz-rendering7). I tested adding a segmentation camera to the actors world and I don't see the broken meshes.