search

tranluan / Nonlinear_Face_3DMM

Source code for "Nonlinear 3D Face Morphable Model"
http://cvlab.cse.msu.edu/project-nonlinear-3dmm.html
Apache License 2.0
676 stars 123 forks source link

Pretrained model #57

Open yuellong opened 4 years ago

yuellong commented 4 years ago

Does anyone successfully run the pretrained model? I was receiving unreasonable outputs.

FYI, I am using ubuntu 16.04 with python2.7.18 and tensorflow 1.13.1.

xingmimfl commented 4 years ago

seems input is 224x224, and the ouput is not very good

ibrahimqreet commented 4 years ago

@yuellong Thank you can you help me turn it on? I am trying to run it on Google Collab? Please can you please guide me with the required steps

cassiePython commented 4 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

ibrahimqreet commented 4 years ago

Can you tell me how you were able to run the pre-trained model, and what steps you took, please @cassiePython

HOMGH commented 4 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

huyanfei-cqupt commented 3 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

HOMGH commented 3 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face

            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5

            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)

            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here: https://github.com/tranluan/Nonlinear_Face_3DMM/blob/662098a602d542c3505cd16ba01dd302f33eeee8/model_non_linear_3DMM_proxy.py#L445

huyanfei-cqupt commented 3 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face

            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5

            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)

            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here:

https://github.com/tranluan/Nonlinear_Face_3DMM/blob/662098a602d542c3505cd16ba01dd302f33eeee8/model_non_linear_3DMM_proxy.py#L445

@HOMGH Thank you very much! I have another question. Do I need to modify the variables G_images_mix_s1a0_300W_vgg and G_images_mix_s0a1_300W_vgg in this code? I once tried to imitate the model_non_linear_proxy.py file to write the G_loss_frecon code, but when I ran it, the system reported an error.

HOMGH commented 3 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face

            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5

            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)

            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here: https://github.com/tranluan/Nonlinear_Face_3DMM/blob/662098a602d542c3505cd16ba01dd302f33eeee8/model_non_linear_3DMM_proxy.py#L445

@HOMGH Thank you very much! I have another question. Do I need to modify the variables G_images_mix_s1a0_300W_vgg and G_images_mix_s0a1_300W_vgg in this code?

No, I don't think so. Actually I didn't get good results after training for 40 epochs which took me around 20 days! So I'm not sure whether this code works or not!

huyanfei-cqupt commented 3 years ago

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face

            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5

            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)

            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here: https://github.com/tranluan/Nonlinear_Face_3DMM/blob/662098a602d542c3505cd16ba01dd302f33eeee8/model_non_linear_3DMM_proxy.py#L445

@HOMGH Thank you very much! I have another question. Do I need to modify the variables G_images_mix_s1a0_300W_vgg and G_images_mix_s0a1_300W_vgg in this code?

No, I don't think so. Actually I didn't get good results after training for 40 epochs which took me around 20 days! So I'm not sure whether this code works or not!

Hi, I would like to ask if the file you are running is model_non_linear.py or model_non_linear_proxy.py? I am now going to try to run the model_non_linear.py file,so I want to ask whether other variables in the file proxy.py need to be modified or discarded. The following is the code I wrote: `G_loss_frecon = tf.zeros(1) if self.is_using_frecon: from vgg_face import vgg_face

            G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask) + 1.0)*127.5
            # G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0) * 127.5
            # G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0) * 127.5
            input_images_300W_vgg = (input_images_300W + 1.0) * 127.5

            input_features, _, _ = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_features, _, _ = vgg_face('vgg-face.mat', G_images_300W_vgg)
            # recon_mix_s1a0_features, _, _ = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            # recon_mix_s0a1_features, _, _ = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)

            layer_names = ['conv1_1', 'conv2_1', 'conv3_1', 'conv4_1',
                           'conv5_1']  # ['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_features[layer_name],
                                           loss_type='l2') / (layer_weight * layer_weight * len(layer_names)) * 2
                # G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name],
                #                            loss_type='l2') / (layer_weight * layer_weight * len(layer_names)) * 2

        g_loss = g_loss + G_loss_frecon`

I am not sure if it is correct, but when I run it, the system prompts an error. Can you help me? thank you!

HOMGH commented 3 years ago

model_non_linear_proxy.py

I ran model_non_linear_proxy.py!

huyanfei-cqupt commented 3 years ago

model_non_linear_proxy.py

I ran model_non_linear_proxy.py!

Have you not tried to run the model_non_linear.py file before?