qianlim
commented
4 years ago Thanks for reporting this as it is potentially a bug where the pretrained group norm variables are not properly loaded at test but just re-initialized (printing out gammas under scope generator_1 will yield constant 1's which are their default init values). I'm working on a fix. But for your question: the params under the scope "generator" are the pretrained ones.
l1346792580123
when I list the pretrained model using tf.train.list_variables, I get results as follows. You can see there are two different group norm weights used by same decoder_resblock such as generator_1/decoder/decoder_resblock_cmr1/group_norm/beta and generator/decoder/decoder_resblock_cmr1/group_norm/beta. I want to transfer the pretrained model from tensorflow to pytorch, so I am not sure which I should transfer to.
[('condition_clo_label/fc1/dense/bias', [8]), ('condition_clo_label/fc1/dense/bias/Momentum', [8]), ('condition_clo_label/fc1/dense/kernel', [4, 8]), ('condition_clo_label/fc1/dense/kernel/Momentum', [4, 8]), ('condition_pose/fc1/dense/bias', [63]), ('condition_pose/fc1/dense/bias/Momentum', [63]), ('condition_pose/fc1/dense/kernel', [126, 63]), ('condition_pose/fc1/dense/kernel/Momentum', [126, 63]), ('condition_pose/fc2/dense/bias', [24]), ('condition_pose/fc2/dense/bias/Momentum', [24]), ('condition_pose/fc2/dense/kernel', [63, 24]), ('condition_pose/fc2/dense/kernel/Momentum', [63, 24]), ('discriminator/prediction_map/weights', [256, 1]), ('discriminator/prediction_map/weights/Momentum', [256, 1]), ('discriminator/shared/conv1/bias', [1, 1, 64]), ('discriminator/shared/conv1/bias/Momentum', [1, 1, 64]), ('discriminator/shared/conv1/weights', [105, 64]), ('discriminator/shared/conv1/weights/Momentum', [105, 64]), ('discriminator/shared/conv2/bias', [1, 1, 64]), ('discriminator/shared/conv2/bias/Momentum', [1, 1, 64]), ('discriminator/shared/conv2/weights', [192, 64]), ('discriminator/shared/conv2/weights/Momentum', [192, 64]), ('discriminator/shared/conv3/bias', [1, 1, 128]), ('discriminator/shared/conv3/bias/Momentum', [1, 1, 128]), ('discriminator/shared/conv3/weights', [192, 128]), ('discriminator/shared/conv3/weights/Momentum', [192, 128]), ('discriminator/shared/conv4/bias', [1, 1, 128]), ('discriminator/shared/conv4/bias/Momentum', [1, 1, 128]), ('discriminator/shared/conv4/weights', [384, 128]), ('discriminator/shared/conv4/weights/Momentum', [384, 128]), ('generator/decoder/1x1-conv/weights', [64, 512]), ('generator/decoder/1x1-conv/weights/Momentum', [64, 512]), ('generator/decoder/decoder_resblock_cmr1/graph_conv/weights', [512, 256]), ('generator/decoder/decoder_resblock_cmr1/graph_conv/weights/Momentum', [512, 256]), ('generator/decoder/decoder_resblock_cmr1/graph_linear_1/weights', [544, 256]), ('generator/decoder/decoder_resblock_cmr1/graph_linear_1/weights/Momentum', [544, 256]), ('generator/decoder/decoder_resblock_cmr1/graph_linear_2/weights', [256, 512]), ('generator/decoder/decoder_resblock_cmr1/graph_linear_2/weights/Momentum', [256, 512]), ('generator/decoder/decoder_resblock_cmr1/graph_linear_input/weights', [544, 512]), ('generator/decoder/decoder_resblock_cmr1/graph_linear_input/weights/Momentum', [544, 512]), ('generator/decoder/decoder_resblock_cmr1/group_norm/beta', [544]), ('generator/decoder/decoder_resblock_cmr1/group_norm/beta/Momentum', [544]), ('generator/decoder/decoder_resblock_cmr1/group_norm/gamma', [544]), ('generator/decoder/decoder_resblock_cmr1/group_norm/gamma/Momentum', [544]), ('generator/decoder/decoder_resblock_cmr1/group_norm_1/beta', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_1/beta/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_1/gamma', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_1/gamma/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_2/beta', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_2/beta/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_2/gamma', [256]), ('generator/decoder/decoder_resblock_cmr1/group_norm_2/gamma/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr2/graph_conv/weights', [512, 256]), ('generator/decoder/decoder_resblock_cmr2/graph_conv/weights/Momentum', [512, 256]), ('generator/decoder/decoder_resblock_cmr2/graph_linear_1/weights', [544, 256]), ('generator/decoder/decoder_resblock_cmr2/graph_linear_1/weights/Momentum', [544, 256]), ('generator/decoder/decoder_resblock_cmr2/graph_linear_2/weights', [256, 512]), ('generator/decoder/decoder_resblock_cmr2/graph_linear_2/weights/Momentum', [256, 512]), ('generator/decoder/decoder_resblock_cmr2/graph_linear_input/weights', [544, 512]), ('generator/decoder/decoder_resblock_cmr2/graph_linear_input/weights/Momentum', [544, 512]), ('generator/decoder/decoder_resblock_cmr2/group_norm/beta', [544]), ('generator/decoder/decoder_resblock_cmr2/group_norm/beta/Momentum', [544]), ('generator/decoder/decoder_resblock_cmr2/group_norm/gamma', [544]), ('generator/decoder/decoder_resblock_cmr2/group_norm/gamma/Momentum', [544]), ('generator/decoder/decoder_resblock_cmr2/group_norm_1/beta', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_1/beta/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_1/gamma', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_1/gamma/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_2/beta', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_2/beta/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_2/gamma', [256]), ('generator/decoder/decoder_resblock_cmr2/group_norm_2/gamma/Momentum', [256]), ('generator/decoder/decoder_resblock_cmr3/graph_conv/weights', [256, 128]), ('generator/decoder/decoder_resblock_cmr3/graph_conv/weights/Momentum', [256, 128]), ('generator/decoder/decoder_resblock_cmr3/graph_linear_1/weights', [544, 128]), ('generator/decoder/decoder_resblock_cmr3/graph_linear_1/weights/Momentum', [544, 128]), ('generator/decoder/decoder_resblock_cmr3/graph_linear_2/weights', [128, 256]), ('generator/decoder/decoder_resblock_cmr3/graph_linear_2/weights/Momentum', [128, 256]), ('generator/decoder/decoder_resblock_cmr3/graph_linear_input/weights', [544, 256]), ('generator/decoder/decoder_resblock_cmr3/graph_linear_input/weights/Momentum', [544, 256]), ('generator/decoder/decoder_resblock_cmr3/group_norm/beta', [544]), ('generator/decoder/decoder_resblock_cmr3/group_norm/beta/Momentum', [544]), ('generator/decoder/decoder_resblock_cmr3/group_norm/gamma', [544]), ('generator/decoder/decoder_resblock_cmr3/group_norm/gamma/Momentum', [544]), ('generator/decoder/decoder_resblock_cmr3/group_norm_1/beta', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_1/beta/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_1/gamma', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_1/gamma/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_2/beta', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_2/beta/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_2/gamma', [128]), ('generator/decoder/decoder_resblock_cmr3/group_norm_2/gamma/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr4/graph_conv/weights', [256, 128]), ('generator/decoder/decoder_resblock_cmr4/graph_conv/weights/Momentum', [256, 128]), ('generator/decoder/decoder_resblock_cmr4/graph_linear_1/weights', [288, 128]), ('generator/decoder/decoder_resblock_cmr4/graph_linear_1/weights/Momentum', [288, 128]), ('generator/decoder/decoder_resblock_cmr4/graph_linear_2/weights', [128, 256]), ('generator/decoder/decoder_resblock_cmr4/graph_linear_2/weights/Momentum', [128, 256]), ('generator/decoder/decoder_resblock_cmr4/graph_linear_input/weights', [288, 256]), ('generator/decoder/decoder_resblock_cmr4/graph_linear_input/weights/Momentum', [288, 256]), ('generator/decoder/decoder_resblock_cmr4/group_norm/beta', [288]), ('generator/decoder/decoder_resblock_cmr4/group_norm/beta/Momentum', [288]), ('generator/decoder/decoder_resblock_cmr4/group_norm/gamma', [288]), ('generator/decoder/decoder_resblock_cmr4/group_norm/gamma/Momentum', [288]), ('generator/decoder/decoder_resblock_cmr4/group_norm_1/beta', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_1/beta/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_1/gamma', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_1/gamma/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_2/beta', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_2/beta/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_2/gamma', [128]), ('generator/decoder/decoder_resblock_cmr4/group_norm_2/gamma/Momentum', [128]), ('generator/decoder/decoder_resblock_cmr5/graph_conv/weights', [128, 64]), ('generator/decoder/decoder_resblock_cmr5/graph_conv/weights/Momentum', [128, 64]), ('generator/decoder/decoder_resblock_cmr5/graph_linear_1/weights', [288, 64]), ('generator/decoder/decoder_resblock_cmr5/graph_linear_1/weights/Momentum', [288, 64]), ('generator/decoder/decoder_resblock_cmr5/graph_linear_2/weights', [64, 128]), ('generator/decoder/decoder_resblock_cmr5/graph_linear_2/weights/Momentum', [64, 128]), ('generator/decoder/decoder_resblock_cmr5/graph_linear_input/weights', [288, 128]), ('generator/decoder/decoder_resblock_cmr5/graph_linear_input/weights/Momentum', [288, 128]), ('generator/decoder/decoder_resblock_cmr5/group_norm/beta', [288]), ('generator/decoder/decoder_resblock_cmr5/group_norm/beta/Momentum', [288]), ('generator/decoder/decoder_resblock_cmr5/group_norm/gamma', [288]), ('generator/decoder/decoder_resblock_cmr5/group_norm/gamma/Momentum', [288]), ('generator/decoder/decoder_resblock_cmr5/group_norm_1/beta', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_1/beta/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_1/gamma', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_1/gamma/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_2/beta', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_2/beta/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_2/gamma', [64]), ('generator/decoder/decoder_resblock_cmr5/group_norm_2/gamma/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr6/graph_conv/weights', [128, 64]), ('generator/decoder/decoder_resblock_cmr6/graph_conv/weights/Momentum', [128, 64]), ('generator/decoder/decoder_resblock_cmr6/graph_linear_1/weights', [160, 64]), ('generator/decoder/decoder_resblock_cmr6/graph_linear_1/weights/Momentum', [160, 64]), ('generator/decoder/decoder_resblock_cmr6/graph_linear_2/weights', [64, 128]), ('generator/decoder/decoder_resblock_cmr6/graph_linear_2/weights/Momentum', [64, 128]), ('generator/decoder/decoder_resblock_cmr6/graph_linear_input/weights', [160, 128]), ('generator/decoder/decoder_resblock_cmr6/graph_linear_input/weights/Momentum', [160, 128]), ('generator/decoder/decoder_resblock_cmr6/group_norm/beta', [160]), ('generator/decoder/decoder_resblock_cmr6/group_norm/beta/Momentum', [160]), ('generator/decoder/decoder_resblock_cmr6/group_norm/gamma', [160]), ('generator/decoder/decoder_resblock_cmr6/group_norm/gamma/Momentum', [160]), ('generator/decoder/decoder_resblock_cmr6/group_norm_1/beta', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_1/beta/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_1/gamma', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_1/gamma/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_2/beta', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_2/beta/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_2/gamma', [64]), ('generator/decoder/decoder_resblock_cmr6/group_norm_2/gamma/Momentum', [64]), ('generator/decoder/decoder_resblock_cmr7/graph_conv/weights', [64, 32]), ('generator/decoder/decoder_resblock_cmr7/graph_conv/weights/Momentum', [64, 32]), ('generator/decoder/decoder_resblock_cmr7/graph_linear_1/weights', [160, 32]), ('generator/decoder/decoder_resblock_cmr7/graph_linear_1/weights/Momentum', [160, 32]), ('generator/decoder/decoder_resblock_cmr7/graph_linear_2/weights', [32, 64]), ('generator/decoder/decoder_resblock_cmr7/graph_linear_2/weights/Momentum', [32, 64]), ('generator/decoder/decoder_resblock_cmr7/graph_linear_input/weights', [160, 64]), ('generator/decoder/decoder_resblock_cmr7/graph_linear_input/weights/Momentum', [160, 64]), ('generator/decoder/decoder_resblock_cmr7/group_norm/beta', [160]), ('generator/decoder/decoder_resblock_cmr7/group_norm/beta/Momentum', [160]), ('generator/decoder/decoder_resblock_cmr7/group_norm/gamma', [160]), ('generator/decoder/decoder_resblock_cmr7/group_norm/gamma/Momentum', [160]), ('generator/decoder/decoder_resblock_cmr7/group_norm_1/beta', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_1/beta/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_1/gamma', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_1/gamma/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_2/beta', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_2/beta/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_2/gamma', [32]), ('generator/decoder/decoder_resblock_cmr7/group_norm_2/gamma/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr8/graph_conv/weights', [64, 32]), ('generator/decoder/decoder_resblock_cmr8/graph_conv/weights/Momentum', [64, 32]), ('generator/decoder/decoder_resblock_cmr8/graph_linear_1/weights', [96, 32]), ('generator/decoder/decoder_resblock_cmr8/graph_linear_1/weights/Momentum', [96, 32]), ('generator/decoder/decoder_resblock_cmr8/graph_linear_2/weights', [32, 64]), ('generator/decoder/decoder_resblock_cmr8/graph_linear_2/weights/Momentum', [32, 64]), ('generator/decoder/decoder_resblock_cmr8/graph_linear_input/weights', [96, 64]), ('generator/decoder/decoder_resblock_cmr8/graph_linear_input/weights/Momentum', [96, 64]), ('generator/decoder/decoder_resblock_cmr8/group_norm/beta', [96]), ('generator/decoder/decoder_resblock_cmr8/group_norm/beta/Momentum', [96]), ('generator/decoder/decoder_resblock_cmr8/group_norm/gamma', [96]), ('generator/decoder/decoder_resblock_cmr8/group_norm/gamma/Momentum', [96]), ('generator/decoder/decoder_resblock_cmr8/group_norm_1/beta', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_1/beta/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_1/gamma', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_1/gamma/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_2/beta', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_2/beta/Momentum', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_2/gamma', [32]), ('generator/decoder/decoder_resblock_cmr8/group_norm_2/gamma/Momentum', [32]), ('generator/decoder/fc1/dense/bias', [55168]), ('generator/decoder/fc1/dense/bias/Momentum', [55168]), ('generator/decoder/fc1/dense/kernel', [50, 55168]), ('generator/decoder/fc1/dense/kernel/Momentum', [50, 55168]), ('generator/decoder/outputs/bias', [1, 6890, 3]), ('generator/decoder/outputs/bias/Momentum', [1, 6890, 3]), ('generator/decoder/outputs/weights', [192, 3]), ('generator/decoder/outputs/weights/Momentum', [192, 3]), ('generator/encoder/1x1-conv/weights', [512, 64]), ('generator/encoder/1x1-conv/weights/Momentum', [512, 64]), ('generator/encoder/encoder_conv1/bias', [1, 1, 64]), ('generator/encoder/encoder_conv1/bias/Momentum', [1, 1, 64]), ('generator/encoder/encoder_conv1/weights', [6, 64]), ('generator/encoder/encoder_conv1/weights/Momentum', [6, 64]), ('generator/encoder/encoder_conv2/bias', [1, 1, 64]), ('generator/encoder/encoder_conv2/bias/Momentum', [1, 1, 64]), ('generator/encoder/encoder_conv2/weights', [128, 64]), ('generator/encoder/encoder_conv2/weights/Momentum', [128, 64]), ('generator/encoder/encoder_conv3/bias', [1, 1, 128]), ('generator/encoder/encoder_conv3/bias/Momentum', [1, 1, 128]), ('generator/encoder/encoder_conv3/weights', [128, 128]), ('generator/encoder/encoder_conv3/weights/Momentum', [128, 128]), ('generator/encoder/encoder_conv4/bias', [1, 1, 128]), ('generator/encoder/encoder_conv4/bias/Momentum', [1, 1, 128]), ('generator/encoder/encoder_conv4/weights', [256, 128]), ('generator/encoder/encoder_conv4/weights/Momentum', [256, 128]), ('generator/encoder/encoder_conv5/bias', [1, 1, 256]), ('generator/encoder/encoder_conv5/bias/Momentum', [1, 1, 256]), ('generator/encoder/encoder_conv5/weights', [256, 256]), ('generator/encoder/encoder_conv5/weights/Momentum', [256, 256]), ('generator/encoder/encoder_conv6/bias', [1, 1, 256]), ('generator/encoder/encoder_conv6/bias/Momentum', [1, 1, 256]), ('generator/encoder/encoder_conv6/weights', [512, 256]), ('generator/encoder/encoder_conv6/weights/Momentum', [512, 256]), ('generator/encoder/encoder_conv7/bias', [1, 1, 512]), ('generator/encoder/encoder_conv7/bias/Momentum', [1, 1, 512]), ('generator/encoder/encoder_conv7/weights', [512, 512]), ('generator/encoder/encoder_conv7/weights/Momentum', [512, 512]), ('generator/encoder/encoder_conv8/bias', [1, 1, 512]), ('generator/encoder/encoder_conv8/bias/Momentum', [1, 1, 512]), ('generator/encoder/encoder_conv8/weights', [1024, 512]), ('generator/encoder/encoder_conv8/weights/Momentum', [1024, 512]), ('generator/encoder/fc_mean/dense/bias', [18]), ('generator/encoder/fc_mean/dense/bias/Momentum', [18]), ('generator/encoder/fc_mean/dense/kernel', [55168, 18]), ('generator/encoder/fc_mean/dense/kernel/Momentum', [55168, 18]), ('generator/encoder/fc_var/dense/bias', [18]), ('generator/encoder/fc_var/dense/bias/Momentum', [18]), ('generator/encoder/fc_var/dense/kernel', [55168, 18]), ('generator/encoder/fc_var/dense/kernel/Momentum', [55168, 18]), ('generator_1/decoder/decoder_resblock_cmr1/group_norm/beta', [544]), ('generator_1/decoder/decoder_resblock_cmr1/group_norm/gamma', [544]), ('generator_1/decoder/decoder_resblock_cmr1/group_norm_1/beta', [256]), ('generator_1/decoder/decoder_resblock_cmr1/group_norm_1/gamma', [256]), ('generator_1/decoder/decoder_resblock_cmr1/group_norm_2/beta', [256]), ('generator_1/decoder/decoder_resblock_cmr1/group_norm_2/gamma', [256]), ('generator_1/decoder/decoder_resblock_cmr2/group_norm/beta', [544]), ('generator_1/decoder/decoder_resblock_cmr2/group_norm/gamma', [544]), ('generator_1/decoder/decoder_resblock_cmr2/group_norm_1/beta', [256]), ('generator_1/decoder/decoder_resblock_cmr2/group_norm_1/gamma', [256]), ('generator_1/decoder/decoder_resblock_cmr2/group_norm_2/beta', [256]), ('generator_1/decoder/decoder_resblock_cmr2/group_norm_2/gamma', [256]), ('generator_1/decoder/decoder_resblock_cmr3/group_norm/beta', [544]), ('generator_1/decoder/decoder_resblock_cmr3/group_norm/gamma', [544]), ('generator_1/decoder/decoder_resblock_cmr3/group_norm_1/beta', [128]), ('generator_1/decoder/decoder_resblock_cmr3/group_norm_1/gamma', [128]), ('generator_1/decoder/decoder_resblock_cmr3/group_norm_2/beta', [128]), ('generator_1/decoder/decoder_resblock_cmr3/group_norm_2/gamma', [128]), ('generator_1/decoder/decoder_resblock_cmr4/group_norm/beta', [288]), ('generator_1/decoder/decoder_resblock_cmr4/group_norm/gamma', [288]), ('generator_1/decoder/decoder_resblock_cmr4/group_norm_1/beta', [128]), ('generator_1/decoder/decoder_resblock_cmr4/group_norm_1/gamma', [128]), ('generator_1/decoder/decoder_resblock_cmr4/group_norm_2/beta', [128]), ('generator_1/decoder/decoder_resblock_cmr4/group_norm_2/gamma', [128]), ('generator_1/decoder/decoder_resblock_cmr5/group_norm/beta', [288]), ('generator_1/decoder/decoder_resblock_cmr5/group_norm/gamma', [288]), ('generator_1/decoder/decoder_resblock_cmr5/group_norm_1/beta', [64]), ('generator_1/decoder/decoder_resblock_cmr5/group_norm_1/gamma', [64]), ('generator_1/decoder/decoder_resblock_cmr5/group_norm_2/beta', [64]), ('generator_1/decoder/decoder_resblock_cmr5/group_norm_2/gamma', [64]), ('generator_1/decoder/decoder_resblock_cmr6/group_norm/beta', [160]), ('generator_1/decoder/decoder_resblock_cmr6/group_norm/gamma', [160]), ('generator_1/decoder/decoder_resblock_cmr6/group_norm_1/beta', [64]), ('generator_1/decoder/decoder_resblock_cmr6/group_norm_1/gamma', [64]), ('generator_1/decoder/decoder_resblock_cmr6/group_norm_2/beta', [64]), ('generator_1/decoder/decoder_resblock_cmr6/group_norm_2/gamma', [64]), ('generator_1/decoder/decoder_resblock_cmr7/group_norm/beta', [160]), ('generator_1/decoder/decoder_resblock_cmr7/group_norm/gamma', [160]), ('generator_1/decoder/decoder_resblock_cmr7/group_norm_1/beta', [32]), ('generator_1/decoder/decoder_resblock_cmr7/group_norm_1/gamma', [32]), ('generator_1/decoder/decoder_resblock_cmr7/group_norm_2/beta', [32]), ('generator_1/decoder/decoder_resblock_cmr7/group_norm_2/gamma', [32]), ('generator_1/decoder/decoder_resblock_cmr8/group_norm/beta', [96]), ('generator_1/decoder/decoder_resblock_cmr8/group_norm/gamma', [96]), ('generator_1/decoder/decoder_resblock_cmr8/group_norm_1/beta', [32]), ('generator_1/decoder/decoder_resblock_cmr8/group_norm_1/gamma', [32]), ('generator_1/decoder/decoder_resblock_cmr8/group_norm_2/beta', [32]), ('generator_1/decoder/decoder_resblock_cmr8/group_norm_2/gamma', [32]), ('loss/total_loss/add_4/ExponentialMovingAverage', []), ('loss/total_loss/add_5/ExponentialMovingAverage', []), ('training/global_step', [])]