boschresearch / OASIS

Official implementation of the paper "You Only Need Adversarial Supervision for Semantic Image Synthesis" (ICLR 2021)
GNU Affero General Public License v3.0
326 stars 54 forks source link

The label of output by the discriminator is inconsistent with the label of the dataset #34

Closed Wzj-zju closed 4 months ago

Wzj-zju commented 2 years ago

Hi, I'm trying to use OASIS discriminator with my own dataset's semantic segmantation. In short, my ground truth onehot segmantation has 3 channels, and it is shaped like: channel 0: background, channel 1: human, and channel 2 for the fake pixel label as paper said. But when calculate cross entropy loss, the target label seems to have been processed into an one channel label like: 1:background, 2:human whenis_real=Trueor 0: fake when is_real=False. The model still works properly, but the predicted semantic segmentation display order is not quite correct when visualizing. Can you please tell me if my process is correct, any suggestions would be very helpful, thanks.

SushkoVadim commented 2 years ago

Hi,

The fake class should not exist in your dataset. Fake class is used for generated images, but not for real images from the dataset. Thus, your dataset is likely to have channels: 0 - background, 1 - human. (N=2) The output of the discriminator has N+1=3 channels (as paper said). The fake class is assigned id=0 for simplicity in our implementation. Thus it becomes 0 - fake, 1 - background, 2 - human.

If the model works properly, then it is probably fine, and no fix is required.