i got zero array from 'recon_img' in .mat file

[quhb2455]

environment : ubuntu 16.04 cuda 9.0 cudnn 7.6.4 tensorflow-gpu 1.12.0 (installed by pip)

everythis is okay when i follow getting started but when i run demo, i got error : InvalidArgumentError (see above for traceback): RasterizeTriangles expects vertices to have shape (-1, 3). so i installed mesh_renderer using pip install mesh_renderer and then the error is solved. also running demo.py is successfully.

i open the .mat file using scipy on jupyter notebook.

mat = scipy.io.loadmat('vd102')
mat['recon_img']
array([[[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        ...,
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]],

       [[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        ...,
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]],

       [[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        ...,
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]],

       ...,

       [[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        ...,
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]],

       [[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        ...,
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]],

       [[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        ...,
        [0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]]], dtype=float32)

the other keys have right values. only 'recon_img' is wrong.

is it right way to open the .mat file on python? and how to get right value of 'recon_img' ??

[YuDeng]

It seems that something is wrong with tf_mesh_renderer. You can try to remove the installed mesh_renderer and follow the instruction in this repo to use the pre-compiled .so library and check if it works.

Our code builds on an older version of tf_mesh_renderer which is not compatible with its latest version. I think that is why you get the InvalidArgumentError.

[quhb2455]

Hi @YuDeng , i really thank you for answering my question.

i have tried to use precompiled .so file before install mesh_renderer package. when i tried it i gotInvalidArgumentError (see above for traceback): RasterizeTriangles expects vertices to have shape (-1, 3). this error and can't solve this problem without install mesh_renderer. if you know about this error, could you let me know how to solve it?

[quhb2455]

InvalidArgumentError (see above for traceback): RasterizeTriangles expects vertices to have shape (-1, 3). this error comes with wrong tf_mesh_renderer. first i follow up the getting started and than i got above error. so i download tf_mesh_renderer from here , remove original tf_mesh_renderer and move downloaded tf_mesh_renderer from above link to under ./Deep3DFaceReconstruction and do a bazel test ... same as getting started. after that i got .so file with several error. but the several error can be ignored. after bazel build, Doing same as getting started.

running Demo.py is okay but running train.py, i had Segmentation fault (core dumped) error. For solving this error, i change my cuDNN versing from 7.2.1 to 7.3.1. here is refer. i begin from follow No3. Install cudnn.

But 'recon_img' is still zero array

[quhb2455]

self answer here is refer. https://github.com/microsoft/Deep3DFaceReconstruction/issues/72#issuecomment-763864508

recon_img = mat["recon_img"]
recon_img = recon_img.astype(np.uint8)
recon_img = Image.fromarray(recon_img)
recon_img = recon_img.convert("RGB")
recon_img.save('recon.jpg')

it works for me.