Run on Windows

[ChairManMeow-SY]

Thanks for authors to share this excellent job!

I just successfully ran the demo on Windows 10. It is painful to search the documents and solve some problems... Here I record it to help anyone who want to run it on windows too.

My env OS: windows 10 GPU: GTX 3080 Ti CPU: Intel 12th Python with conda env

Step 1. Install packages in requirements.txt

The only thing you should pay attention to is the transformers version. Use the following command:

pip install transformers==4.6.1

Step 2. Install pyopengl Do not use 'pip install pyopengl' directly because it offers a package lacking some libraries (It is amazing but true...). Download the .whl file from here and install it with:

pip install <downloaded file>

Note that do not install the opengl before you have installed the pyrender!! This will uninstall your pyopengl and install a pypi version, which lacks the necessary libraries.

Step 3. Install boost Just do with the google instructions.

Step 4. Install mesh Download the mesh from here and following its readMe, i.e.

pip install --no-deps --install-option="--boost-location=<path_to_your_boost>" --verbose --no-cache-dir .

Step 5. Install ffmpeg You can find a pre-compiled version from ffmpeg website. Just double click it.

Step 6. Revise the demo.py Finally we have install all the necessary packages. But if you run the demo directly, it will throw an error: can not find osmesa. The easist way to solve it is to comment this line:

os.environ['PYOPENGL_PLATFORM'] = 'osmesa' # egl

The pyrender will use pyglet for off-screen rendering. Some explainations can be found here.

Then replace the render_sequence(args) funciton with the following code

def render_sequence(args):
    wav_path = args.wav_path
    test_name = os.path.basename(wav_path).split(".")[0]
    predicted_vertices_path = os.path.join(args.result_path,test_name+".npy")
    if args.dataset == "BIWI":
        template_file = os.path.join(args.dataset, args.render_template_path, "BIWI.ply")
    elif args.dataset == "vocaset":
        template_file = os.path.join(args.dataset, args.render_template_path, "FLAME_sample.ply")

    print("rendering: ", test_name)

    template = Mesh(filename=template_file)
    predicted_vertices = np.load(predicted_vertices_path)
    predicted_vertices = np.reshape(predicted_vertices,(-1,args.vertice_dim//3,3))

    output_path = args.output_path
    if not os.path.exists(output_path):
        os.makedirs(output_path)

    num_frames = predicted_vertices.shape[0]
    tmp_file='./output.mp4'

    writer = cv2.VideoWriter(tmp_file, cv2.VideoWriter_fourcc(*'mp4v'), args.fps, (800, 800), True)
    center = np.mean(predicted_vertices[0], axis=0)

    for i_frame in range(num_frames):
        render_mesh = Mesh(predicted_vertices[i_frame], template.f)
        pred_img = render_mesh_helper(args,render_mesh, center)
        pred_img = pred_img.astype(np.uint8)
        writer.write(pred_img)

    writer.release()
    file_name = test_name+"_"+args.subject+"_condition_"+args.condition
    video_fname = os.path.join(output_path, file_name+'.mp4')
    #cmd = ('ffmpeg.exe' + ' -i {0} -pix_fmt yuv420p -qscale 0 {1}'.format(tmp_video_file.name, video_fname)).split()
    cmd='ffmpeg -i %s -i %s %s' %(tmp_file,wav_path,video_fname)
    os.system(cmd)

Here are two modifications:

1. use a string file name instead of the temp file because the ffmpeg will throw a permission error.
2. use ffmpeg to put the wav and video together. This can help check the algorithm performance.

Then enjoy it now : )

[ShaojunBian]

Thanks so much! your record helps me a lot!

[ChairManMeow-SY]

hello, when i do the step 4, i have the problem "Running setup.py install for psbody-mesh did not run successfully.", how can i solve it ?

Hi yichen, I notice it may be a compiling error:

D:\git_data\mesh\mesh\src\AABB_n_tree.h(9): fatal error C1083: Cannot open include file: 'CGAL/AABB_tree.h': No such file or directory

The compiler can not find the CGAL head file. It is strange because the package should contain a CGAL package in the folder D:\git_data\mesh\mesh\thirdparty\ .

Please check it or you may decode it manually.

[zjuyichen]

hello, when i do the step 4, i have the problem "Running setup.py install for psbody-mesh did not run successfully.", how can i solve it ?

Hi yichen, I notice it may be a compile error:

D:\git_data\mesh\mesh\src\AABB_n_tree.h(9): fatal error C1083: Cannot open include file: 'CGAL/AABB_tree.h': No such file or directory

The compiler can not find the CGAL head file. It is strange because the package should contain a CGAL package in the folder D:\git_data\mesh\mesh\thirdparty\ .

Please check it or you may decode it manually.

thank your response, I took a very stupid mistake: i didn't git checkout the right branch……

[ChairManMeow-SY]

hello, when i do the step 4, i have the problem "Running setup.py install for psbody-mesh did not run successfully.", how can i solve it ?

Hi yichen, I notice it may be a compile error: D:\git_data\mesh\mesh\src\AABB_n_tree.h(9): fatal error C1083: Cannot open include file: 'CGAL/AABB_tree.h': No such file or directory The compiler can not find the CGAL head file. It is strange because the package should contain a CGAL package in the folder D:\git_data\mesh\mesh\thirdparty\ . Please check it or you may decode it manually.

thank your response, I took a very stupid mistake: i didn't git checkout the right branch……

Great to know the problem solved!

[Langwenchong]

hello，firstly thank your for your record,it helped me so much, but i found there is no sound in my output.mp4,have you ever encountered this problem? If so, can I ask you how to solve it

[ChairManMeow-SY]

hello，firstly thank your for your record,it helped me so much, but i found there is no sound in my output.mp4,have you ever encountered this problem? If so, can I ask you how to solve it

hi there~可以看一下step 6的函数最后几行：

cmd='ffmpeg -i %s -i %s %s' %(tmp_file,wav_path,video_fname)

可以参考这个cmd，在python脚本里面对音视频组装，或者在命令行里面手动调用ffmpeg。tmp_file 指的是网络的输出文件， wav_path是输入的音频文件，video_fname是你希望保存的文件名。

[Langwenchong]

I got the program running, thank you so much😁