tesla123465
commented
1 year ago Please see figure 8 of the above paper for examples of synthesized film grain.
The AV1 video codec analyzes film grain at encoding time to generate parameters for film grain synthesis at decoding time. A similar process could be used for face swaps where film grain is analyzed in the source face to generate parameters for film grain synthesis in the generated face.
joolstorrentecalo
Video recordings often have film grain. Generated faces usually have no film grain, causing face swaps to stand out, particularly if the source video has a noticeable amount of film grain.
One solution to this is to intentionally add film grain during merging so that the generated faces have as much film grain as the rest of the scene. Adding film grain is a common technique employed by video codecs to combat signal loss and make video playback look like the video source. Here is an example of film grain synthesis employed by the AV1 codec: https://norkin.org/pdf/DCC_2018_AV1_film_grain.pdf
Note that this is the opposite of the denoising option that exists in the merger. Rather than removing noise, I am proposing that we add film grain to improve realism.