"Empty" frontal rendering with constant colour with the BFM2009

[zhixiongzuo]

Dear Huber, Thanks for your work. When I use BFM2009 in eos fit-model-multi, I get an abnormal frontal png.(I have a blendershape file belong with BFM2009,as well as UV-coordinates. )There is nothing in frontal png.Also the isomap is different from yours maybe because of different UV-coordinates. Below is frontal and isomap png.

I want to know what is wrong with it,what can i do to solve the problem. I have noticed that there is only 18 landmarks in ibug_to_bfm2009.txt,and 16 landmarks in bfm2009_contours.json, but 50 landmarks in ibug_to_sfm.txt,and 34 landmarks in sfm_contours.json,should I add landmarks in these two files,and how to add?

Thanks

[patrikhuber]

Hi,

It looks like the texture map is correctly extracted, but the frontal rendering is just pink. That's a bit weird. It looks like the colour might be from somewhere on the skin actually. Perhaps try fit-model-simple, and/or just step through the code?

The fact that the texture map is upside-down is fine, it's usually if the program where you created them has a different image origin (e.g. top-left vs bottom-left). If you're bothered by it, you can flip the vertical axis of the uv coords.

As to the number of landmarks/mappings, even 5-10 would be good enough for a rough fit.

[zhixiongzuo]

Thanks for your timely reply,I will step into the code instead of the number of landmarks.I have another problem about fit-model-multi, it cost 2 minutes for fit-model-multi with sfm-29587 to fit 3pic(960*540),as for bfm2009 it's 7 minutes. (My CPU is E5-2620 v2) How can I optimize the fitting time? Thanks for your time.

[patrikhuber]

Regarding the runtime, you are probably compiling a debug build. Compile a release build. And the answer to runtime problems/questions is always: Run a profiler (VS has a nice one, there's others on Linux) to see where it is slow.

[zhixiongzuo]

Yeah it is.Thanks for your advice.

[zhixiongzuo]

I figure out the problem is caused by too large Vertex coordinates of shapeMU in BFM2009, so the calculated current_pose's scale will be much smaller (E.g 0.01,which is 10 in eos-3448),and the current_pose's tx and ty will be much larger, so the front rendering will be empty. So I solve this by rescale shapeMU in convert-bfm2009-to-eos.py

Thanks for your help again.

[patrikhuber]

Yes, the BFM2009 is indeed in different units.

eos is "unit-agnostic" though, and just works with whatever units a model is in. So probably you just used a "wrong" (too large) scaling factor when rendering the frontal image, and the better solution would probably be to just adjust that scaling when rendering, instead of rescaling the model. But both would work of course.

Glad you figured it out!

[zhixiongzuo]

Yeah, rescaling the model is not an ideal solution, because it cost me 3 times than before to reconstruction a 3D face(maybe because of floating point calculation or something else I have no idea). I will try to adjust scaling when rendering, thanks for your advice. best.

[heruiyuan]

@zhixiongzuo Regarding the fit-model-multi, can you show your step or some details to reconstruct a 3D face? I just get some runtime error in using fit-model-multi : "Error opening given file: ../share/model_contours.json"