Attribute Classifier for Editing Accuracy/Error

[tegillis]

I'm curious what you used for the attribute classifier to measure the attribute editing accuracy and preservation error. Also do you have any plans to release this trained model? Thanks.

[LynnHo]

The codes are messed up and I currently have little time to arrange them to make it easy to be used by others. However, here is the code for the classifier network structure.

from __future__ import division
from __future__ import print_function
from __future__ import absolute_import

import tflib as tl
import tensorflow as tf
import tensorflow.contrib.slim as slim

from functools import partial

conv = partial(slim.conv2d, activation_fn=None, weights_regularizer=slim.l2_regularizer(1e-4))
fc = partial(tl.flatten_fully_connected, activation_fn=None, weights_regularizer=slim.l2_regularizer(1e-4))
relu = tf.nn.relu
batch_norm = partial(slim.batch_norm, decay=0.9, scale=True, epsilon=1e-5, updates_collections=None)
pool = partial(slim.max_pool2d, kernel_size=2, stride=2)

def classifier(x, att_dim=40, dim=32, reuse=True, training=True):
    bn = partial(batch_norm, is_training=training)
    conv_bn_relu = partial(conv, normalizer_fn=bn, activation_fn=relu, biases_initializer=None)

    with tf.variable_scope('classifier', reuse=reuse):
        y = conv_bn_relu(x, dim * 1, 3, 1)
        y = conv_bn_relu(y, dim * 1, 3, 1)
        y = pool(y)
        y = conv_bn_relu(y, dim * 2, 3, 1)
        y = conv_bn_relu(y, dim * 2, 3, 1)
        y = pool(y)
        y = conv_bn_relu(y, dim * 4, 3, 1)
        y = conv_bn_relu(y, dim * 4, 3, 1)
        y = pool(y)
        y = conv_bn_relu(y, dim * 8, 3, 1)
        y = conv_bn_relu(y, dim * 8, 3, 1)
        y = pool(y)
        y = relu(fc(y, dim * 16))
        logits = fc(y, att_dim)
        return logits

This classifier is for 64x64 images. @tegillis

[tegillis]

Does this mean the attribute experiments were only done on 64x64 generated images?

I am trying to reproduce the results for AttGAN with 128x128 images and I'm getting much lower accuracy for attribute editing than the values reported in the paper.

I am using images outputted by the test script using the well-trained 128x128 model released here: #2 with a VGG16 + BN architecture for the attribute classifier. Thanks.

[LynnHo]

@tegillis Yes, the attribute experiments were done on 64x64 AttGAN (without shortcut layers) since we only compare our method to IcGAN and VAE/GAN in our first version (see the comparisons in the README.md). The latterly added methods (StarGAN etc.) with 128 output are all resized to 64 for testing the accuracy. I haven't uploaded the model of 64x64 because it is trained by the code before the GitHub version, and cannot be used by the current codes.

[tegillis]

Just to clarify, the attribute results shown in figure's 8 and 9 were all done on 64x64 models? I'm confused as to why StarGAN is grouped with VAE/GAN and IcGAN and not Fader Networks and CycleGAN since in Part IV you group StarGAN with Fader Networks, Shen et al and CycleGAN for being a 128x128 model.

[LynnHo]

@tegillis StarGAN, VAE/GAN, IcGAN, and AttGAN are able to handle multiple attributes in a single model, while the other three cannot. The image sizes for these methods depend on their open-source codes.

[LynnHo]

@tegillis I upload a 128 model trained by the GitHub code and its classification performance is similar to the paper. I change the weight of xa__loss_rec 100.0 to 10.0 here, https://github.com/LynnHo/AttGAN-Tensorflow/blob/c385855f9ebf054c72d0bc0c60e66e9d64d70805/train.py#L159

[LynnHo]

Here is an attribute classifier from STGAN which has the same protocol of our AttGAN.