incorrect bounding boxes some classes seadronesee

[fabiopoiesi]

There are bounding boxes of some classes that are smaller than the object. See example below (ignore class name for now, I will open another issue).

These are some zoomed-in cases:

This is the code I am using to visualise this data:

import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle

fns = glob.glob(os.path.join(source_dir, dir, 'labels', '*txt'))
id = 0
for fn in fns:
    imageid = os.path.basename(fn).split('.')[0]
    anns = np.loadtxt(fn, dtype=float)

    img = Image.open(os.path.join(source_dir, dir, 'images', '%s.jpg' % imageid))

    plt.imshow(img)
    ax = plt.gca()
    for i, bbox in enumerate(anns):
        rect = Rectangle((1920 * (bbox[1] - bbox[3]/2),
                          1080 * (bbox[2] - bbox[4]/2)), 1920 * bbox[3], 1080 * bbox[4],
                         linewidth=3,
                         edgecolor='r',
                         facecolor='none')
        plt.text(1920 * (bbox[1] - bbox[3]/2) - 2, 1080 * (bbox[2] - bbox[4]/2) - 2, class_names[int(bbox[0])], fontsize=10, color='r')
        ax.add_patch(rect)
    plt.show()

Can you please check this problem, it apparently occurs only with some classes.

[fabiopoiesi]

I had the impression that the annotation is done only for the part above the water, but then I checked in the real dataset and it is not consistent. Here some examples:

[C-der-Baum]

It says in the second bullet point of the "Things to note" section in the readme:

"Currently the segmentation of objects on water is relatively unintuitive. The segmentation works in such a way that only the part of the object above the water surface is part of the segmentation mask, while the part of the object below the water surface is not part of the segmentation mask. However parts of the objects below the water are still visible. With our current methodolgy of calculating the segmentation data from the depth and stencil buffers it is not possible to extend the segmentation to the parts of the objects below the water surface. However, if one wanted to fix this, there is a water opacity buffer in the rendering pipeline that contains inforamtion about the depth below the water for each pixel (see https://www.adriancourreges.com/blog/2015/11/02/gta-v-graphics-study/ for a detailed explanation of the GTAV rendering pipeline, you can use RenderDoc or NVIDIA Nsight to dissect the rendering pipeline). This buffer would need to be extracted by extending GTAVisionExport-DepthExtractor. Then one could combine this buffer data with the stencil buffer and then get the correct segmentation masks from those."

Unfortunately, that means for swimmers that often only the head is properly annotated even though the body is still visible. Therefore, it is not consistent with the real counterpart. Currently, we do not plan to extend a functionality considering the transparency/opacity of water. Feel free to add a pull. Is it fine for you to close this?

[fabiopoiesi]

ok thanks!