srrichter
commented
2 years ago We use the gt_labels for separate processing streams in the G-buffer encoder. Each of the labels gets a separate branch in the encoder network. It's named gt_labels as we used the ground truth segmentation for the synthetic dataset to derive a segmentation into different materials. The key idea is to provide a segmentation of the synthetic data into different materials with the gt_labels. There is no need for any other semantic meaning or correspondence to classes in some dataset for these gt_labels.
The robust-labels are used to guide the discriminator. To that end, they need to be compatible and as consistent as possible for synthetic and real dataset. You could use ground truth semantic segmentation labels here, but this is costly to annotate for real data and most of the time requires considerable effort for the synthetic data as well. The alternative is a robust segmentation network, which works directly on images and produces approximately consistent semantic segmentations for synthetic and real data. We want the discriminator to judge realism conditioned on semantic classes. So a semantically meaningful segmentation is important for the robust_labels.
lucamarini22
groundtruth labels are manually labeled and robust labels are labeled using a neural net. See page 3 of the paper: "These works use groundtruth annotations for guiding the discriminator. While this provides ground-truth semantic information, it restricts the image collections available for training to densely annotated datasets compatible with the synthetic label maps at hand. In our work, we leverage a robust semantic segmentation network [72] to provide label maps that are approximately consistent for synthetic and real images. Previously,"