Would it be possible to position of the head back to it's place?

[carbendi]

I was wondering ıf it would be possible to re use the data that is generated (probably) while tracking the head it one place to put it in it's original place.

use case: after processing a video with animegan2 the head would be in its own place so the original head can be replaced with the animegan version easliy.

right now I can do it by tracking the original head and using the tracking data to put the processed head in place but it is too jumpy...

[bryandlee]

Hi, I don't quite get what you mean by re-using the generated data to put in it's original place. Is it like [crop the head -> model inference -> put back to original image]? or like warp the previous generated frame using optical flow?

[carbendi]

Hey sorry for the confusion,

So If I understand correctly the script detects the face, stabilize and than process it. And the reseult is like in this video. https://youtu.be/fZ3AqJwcPhU

When I want to replace the real face ( in the target video) with the toonified one, the toonified one is jumpy (becouse of the face detect and stabilization)

I was wondering if it is possible to use the stabilization data that the script generated by head tracking in the first place, reverse it and apply to the final image? So it will have the original movement and size and it will follow the head as it was in the original video.

Hope this makes sense :)

[bryandlee]

Actually you don't need to crop & align the head. Instead, you can simply resize the image:

1. get a video frame
2. run face detector on the video frame
3. compute the face size x
4. resize the video frame so that the resulting face size becomes about 400px (which means the resize ratio = 400 / x)
5. apply model to the resized image, and resize the output image back to its original size
6. go back to 1 and process next frame