Weird results

[AaronJackson]

Hi Patrik,

I am trying to run eos with the higher resolution models and took 40 minutes to produce the result below? How long should it normally take for this the high resolution model, and do you have any idea why this is happening? I am using the master branch which I pulled yesterday.

Thanks!

Oh, btw, it works fine with the lower 3k model.

[patrikhuber]

Hi,

Can you try increasing the regularisation quite a lot? Maybe try lambda = 2000?

In any case it should definitely not take 40 minutes. Did you do a Release build with optimisations on? (But even in a debug build it shouldn't take 40mins).

[patrikhuber]

I just ran it on the example image (the revision is more or less around tag v0.12.1, but shouldn't matter much):

fit-model.exe  -m "eos\share\sfm_shape_29587.bin" -b "eos\share\expression_blendshapes_29587.bin" -e "eos\share\sfm_29587_edge_topology.json"
Finished fitting and wrote result mesh and isomap to files with basename "out".

It took around a minute and produced a nice result. I didn't change any of the default parameters of fit-model.

I think the speed issue is an issue with your setup. The 29k model will be slower (1sec vs 1min seems reasonable), mainly because of the contour vertex computation probably. Apart from that, as mentioned, a higher regularisation (lambda) might be necessary with the bigger model, I welcome any input with regards to that.

[AaronJackson]

Hmm, I too am using default parameters without any changes. Although I do agree, it does appear as though I am doing something odd. I am using Linux, I really doubt this is the issue but I'll try and find a Windows PC to test it on tomorrow. Thanks for taking the time to test that.

[patrikhuber]

I just tested on the example image in my Linux Mint 18.1 VM (more or less like Ubuntu 14.04), tag v0.12.1, default gcc from toolchain repo (5.4.0), and everything on default settings (obviously a release build - don't forget to set CMAKE_BUILD_TYPE=Release, as with any project. Maybe you forgot that? I see this is not mentioned in our Readme, I'll add that.):

$ time ./fit-model -m ../share/sfm_shape_29587.bin -b ../share/expression_blendshapes_29587.bin -e ../share/sfm_29587_edge_topology.json 
Finished fitting and wrote result mesh and isomap to files with basename "out".

real    0m39.136s
user    0m38.876s
sys 0m0.228s

And the result looks good. The fitting is not as good as with the 3448 model though, so I think what I said earlier about regularisation is probably true: The larger model needs a higher value.

I suggest:

1) Fix your speed issue, maybe checkout v0.12.1 just to be sure

2) Once fixed, try on the example image. Does it work there? Try to adjust the regularisation on your picture. Also, if you send me your pic with .pts landmarks, I can quickly run it.

[patrikhuber]

Added hint to readme in 3cde05cca0cd086f8c14b54d0375c0fa5388a760. Thanks for giving me that thought, it's a good idea to mention that for people who are not too familiar with CMake.

PS: I admire your patience for waiting 40 minutes!! :-D

[AaronJackson]

It seems I forgot to do a release build. This makes it significantly faster (about one minute). The fitting is quite good on the included example, but I still have very similar output on the AFLW image. Although this is AFLW2000, which probably has some normalisation applied. Maybe this explains the difference between our output. I'll check this and increasing the regularisation. As usual, thanks again for your help. :)

Edit: Yep, increasing lambda helped a lot!

[patrikhuber]

Cool! Let me know if increasing the regularisation doesn't solve the weird shape issue. (I'm not sure I understand what you mean regarding AFLW2000 and normalisation).

Ah I see you've edited and increasing lambda helped! Great :-)

[AaronJackson]

I still find that on some very large poses, such as 70+ degrees yaw, the faces become slightly non-face like. The most reliable value I have found has been lambda 5000, but not really understanding the parameter, I don't know if this is too large.

[patrikhuber]

I would be really interested to get some of these extreme pose images with pts landmarks to have a look and try a bit of stuff by myself. Would you be in a position to share these? (E.g., does it occur with the 3448 model as well?)

Lambda is just the regularisation parameter, how much the shape coefficients get pulled towards the mean - you can read about it in more details in nearly every other 3DMM fitting paper :-) (e.g. they talk about it, in fact they use a box constraint in some equations instead of a L2 regulariser).