akanazawa / hmr

Project page for End-to-end Recovery of Human Shape and Pose
Other
1.54k stars 395 forks source link

Render bugs occurs when vertices are more than 20000 #21

Closed lvvvmd closed 5 years ago

lvvvmd commented 5 years ago

Hi Professor, I am a master from Zhejiang University working on human face reconstruction and I tried to using your code for rendering face mesh. But when I replace the human body vertex and face index in your code with BFM model which has almost 50000 vertices and 90000 faces, then the renderer doesn't work well and some faces whose vertex indices are greater than 20000(not precise, maybe 19000) become black. mesh_rst0_0 And I did other experiments which also showed that when vertices are more than 20000(not precise, maybe 19000) and some faces' using these large index vertices whill cause this bug. For example, one experiment is as follows. At first, I split the face mesh into 3 parts each contains no more than 17000 vertices and no more than 32500 faces, and render these 3 parts recursively by calling your rendering interface in a loop and seting the previous output as background of the current step. So I can render these 3 parts together on one image. result is as follows: mesh_rst0_0

And I am sure that the face mesh is correct and I haven't done any modification on the code except replace the mesh vertices and faces with my BFM face model.
So can you give me some advice and can you do some experiment on the large mesh with more than 20000 vertices?
Thank you very much!
blzq-sh commented 5 years ago

I think the original code for the renderer is at https://github.com/mattloper/opendr, so you can try to ask there, although the author doesn't seem to be responding.

akanazawa commented 5 years ago

Hi,

Yes this is an issue with OpenDr. There are some other opensource differentiable renderers now, like NMR and tf_mesh_renderer.

Although 20k> vertices are a lot for these projects in my experience. I recommend doing differentiable rendering things in a lower resolution mesh. And use a low res -> high res mapping to recover the high resolution version like this paper

Best,

Angjoo