cyberbotics / webots

Webots Robot Simulator
https://cyberbotics.com
Apache License 2.0
3.31k stars 1.72k forks source link

Camera can see through some objects unexpectedly #6427

Open PeterJCLaw opened 1 year ago

PeterJCLaw commented 1 year ago

Describe the Bug

I've hit a couple of cases where a camera can see recognition objects "through" other objects which ought to be opaque (credit to @Adimote for pinning down these cases).

In this cut-down world we have a number of recognition objects which are playing the part of fiducial markers. They're attached to the outside of other objects which are themselves not intended to be directly recognisable and should be opaque. However in a couple of cases the "markers" can be seen through other objects: image

Of the three objects being picked up, two shouldn't be:

The middle one (the white square on the grey bucket) is fine -- this object is nearer the camera than anything else and is behaving as expected.

Note that the brown cube has recognition objects on all six faces, yet only the one facing the camera is picked up. Even the one which is actually partly visible out the top of the bucket is (correctly) occluded.

This is with the default occlusion setting (i.e: 1, which I believe uses centre-points). While debugging this I tried the other values:

Steps to Reproduce

I've put the world (camera-see-through-things.wbt) and its trivial controller in https://gist.github.com/PeterJCLaw/505d36575cddbd1cb1bc21adf489d3a1. I've cut this down significantly from our original world, though it may be possible to cut it down further -- I haven't pushed for an extreme cut-down. Running it in Webots should result in seeing the same thing as shown in the screenshot above.

Expected behavior

Ideally only the nearest of the recognition objects should be picked up -- the one on the outside of the grey bucket ("S3" in the world file) which is on the side nearest the camera.

System

Additional context

PeterJCLaw commented 1 year ago

Oh, forgot to add -- there's another case we hit where if two of the brown cubes are stacked then the camera can sometimes see the "marker" from the topwards face of the bottom one in some cases where it isn't actually visible in the picture. I've added that as another world in the same gist (camera-see-through-more-things.wbt), though I'm assuming it's going to be similar to the cases already described above.

image