JingyuanYY / EmoGen

This is the official implementation of 2024 CVPR paper "EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models".
56 stars 7 forks source link

Mean and std of emotion space #3

Closed DarrenZhaoFR closed 6 months ago

DarrenZhaoFR commented 7 months ago

Hello, interesting work! Can you maybe explain more about the mean and std of a emotion space, how do you calculate them? In my understanding, there should be a trained classifier from stage 1, outputing classification results (N,8), as shown in model.py

class BackBone(nn.Module):
    def __init__(self, ):
        super().__init__()
        self.cnn = models.resnet50(pretrained=True)

        self.backbone = nn.Sequential(*list(self.cnn.children())[:-2])
        self.flaten = nn.Sequential(nn.AvgPool2d(kernel_size=7), nn.Flatten())
        self.fc_1 = nn.Linear(2048, 768)
        self.fc_2 = nn.Sequential(
            nn.ReLU(),
            nn.Dropout(0.5),
            nn.Linear(768, 8)
        )

    def forward(self, x):
        x = self.backbone(x)
        x = self.flaten(x)
        x = self.fc_1(x)
        x = self.fc_2(x)
        return x

I'm guessing the mean and std are calculated from self.fc_1? Thanks!

fengjw0909 commented 6 months ago

Yes, we used the output of the trained BackBone module's self.fc_1 as the data for calculating the mean and std.