search

carla-simulator / carla

Open-source simulator for autonomous driving research.
http://carla.org
MIT License
10.82k stars 3.48k forks source link

Instance Segmentation does not show terrain #6905

Open yoakim82 opened 7 months ago

yoakim82 commented 7 months ago

CARLA version: 0.9.14 (custom build with included custom objects) At least Town01 and Town03 affected, probably all maps.

Platform/OS: Ubuntu 22.04 LTS

Problem you have experienced:

When creating snapshots of a scene with both depth, rgb and segmentation sensors, there is an unexpected behavior in the segmentation view of the scene. Note I'm referring to the instance segmentation sensor here. We use the instance segmentation to capture tight bounding boxes for our objects of interest, but when an object is spawned behind a hill or piece of terrain, the segmentation view will show the object even though it is 100% occluded in the rgb view. This totally ruins the purpose of having semantic segmentation.

terrain_traffic_light_rgb terrain_traffic_light_inst_seg

What you expected to happen: I expect the objects being occluded by the terrain to not show up in the segmentation view, which seem to be the case for other occluding objects (e.g. trees/vegetation). To clarify, the bounding boxes shown in the rgb image are generated only from information in the instance segmentation image.

Steps to reproduce: Setup a triplet of sensors and navigate to a place in the map like in our enclosed scene.

nikky4D commented 7 months ago

did you have this solved?

yoakim82 commented 7 months ago

No, I did not. My guess is it's related to how the town is created and how the terrain is labeled semantically.

But I have no clue as to how to fix this. Anyone with deeper UE4 skills is welcome to comment on this.

Yongzhaooo commented 3 months ago

I met the same situation and it is still not fixed now : ( image