afruehstueck
commented
1 year ago Thank you! The head exchange is a bit more experimental and, depending on the video, can be more tricky to get to work nicely, which is why we did not provide a demo for this. However, you can achieve results using the following steps:
Use Case I: Use inverted video of person's head on top of different video
(1) Fine-tune generator G on a set of images of your target person A
(2) Invert the video containing desired head motion of person A, using generator G
(3) Use video of different person B as target video - must have the same number of video frames as video A. If the head and/or hair sizes are significantly different, some care must be taken (e.g. change the border size) to make the stitching work nicely
Use Case II: Head of target person should follow motion of different video
(1) Fine-tune generator G_A on a set of images of your target person A
(2) Fine-tune another generator G_B on a set of images of the person B shown in your target video
(3) Invert the video of your choice, showing person B using generator G_B
(4) Run our editing method (you may have to slightly edit the code to specify two different savepoints) and mix and match the desired properties from above steps, e.g.
generator G_A + corresponding w_person, but using w_offsets_video from video B
Hope this helps!
Hi,
Congrats on this great research. The video demo shows the head exchange capabilities, but I can't seem to find any code or documentation about how that is done. Is there a Jupyter notebook that you could upload for that demo?
thanks!