search

AaronJackson / vrn

:man: Code for "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression"
http://aaronsplace.co.uk/papers/jackson2017recon/
MIT License
4.52k stars 747 forks source link

About the alignment of background-image and 3dmodel in the online demo #140

Closed wtongping closed 4 years ago

wtongping commented 4 years ago

Dr AaronJackson: Thanks of your excellent works! My question is that how do you align the 3d face to background-image in the online demo. I want to project the 3d face to 2d background-image, but I can`t align well. Your demo: QQ截图20200109173928 My alignment is not as good as yours: 000023_-40

AaronJackson commented 4 years ago

Are you reimplementing? I'm a bit confused about what you are showing.

During training you want to make sure that the input face image and the 3D volume are perfectly aligned. If you were to take the orthographic projection of the 3D volume, you would want it to do a good job at segmenting the input face. It looks from your image that there is a bit of a mixed transformation between the two, maybe some scale, definitely some rotation and translation.

wtongping commented 4 years ago

Are you reimplementing? I'm a bit confused about what you are showing.

During training you want to make sure that the input face image and the 3D volume are perfectly aligned. If you were to take the orthographic projection of the 3D volume, you would want it to do a good job at segmenting the input face. It looks from your image that there is a bit of a mixed transformation between the two, maybe some scale, definitely some rotation and translation.

I took the orthographic projection of the 3D volume, and then got a 2D face image of specified angle . After that, I do PerspectiveTransform on background-image with same angle. In the end, I want to add the 2D face image and background-image to get the result as your demo screenshot. I discover there some different between my PerspectiveTransform`s background-image and yours, maybe PerspectiveTransform is not suit. thx for your reply.