search

davidbau / rewriting

Rewriting a Deep Generative Model, ECCV 2020 (oral). Interactive tool to directly edit the rules of a GAN to synthesize scenes with objects added, removed, or altered. Change StyleGANv2 to make extravagant eyebrows, or horses wearing hats.
https://rewriting.csail.mit.edu/
MIT License
536 stars 76 forks source link

How to get v_*i #7

Open wty-ustc opened 3 years ago

wty-ustc commented 3 years ago

In you paper, image I am confused how to get v*i? Through gan inversion optimization to find the vi correspond to x_i？ I can't find in your code, so sorry to bother you.

davidbau commented 3 years ago

So the (k, v) correspondence comes from the "copy-paste" action.

See the function rewrite.ganrewrite.ProgressiveGanRewriter.paste_from_selection. The output variable goal_in corresponds to k and the variable goal-out is v. Note that goal_in is the context_model (layer "L-1") representation of the target context, like a building spire, and the goal_out is the target_model (layer "L") representation of the replacement pattern, like a tree that we want to put in place instead of building spires. Note that we can get both representations through just inference in the GAN: no GAN inversion is needed.