search

deepfakes / faceswap-playground

User dedicated repo for the faceswap project
303 stars 194 forks source link

Faceswap looks just a paste on top of it #94

Closed Crypt1cBunny closed 6 years ago

Crypt1cBunny commented 6 years ago

Expected behavior

I would expect the model being close to realistic.

Actual behavior

The model just pastes a square on top of the source picture. The contrast does not match and you can clearly see it's a cropped out square. The model training weight was below 0.02.

Maybe I'm just missing some options here... How can I make sure the contrast etc is matched and the image is more blending on the other instead of a square placed on top of it?

I'm basically just using the default settings.

Clorr commented 6 years ago

Please post an example so that we can see what kind of problem this is. If the face is badly generated it is likely that there was not enough/varied example for the model to learn a specific case. If it is badly merged, you will likely have to use seamless merge or some options to blend the generated face better on the original.

Crypt1cBunny commented 6 years ago

The face was generated correctly (not blurry). The main issue is that it's cut out very badly and the contrast is totally different. Can you please tell me more on seamless merge and other options?

torzdf commented 6 years ago

Check

python faceswap.py convert -h

for more specific options.

In the first instance I would try increasing the blur. The amount will depend on the source/target frames, but maybe ramp it up to about 30 and go from there. This should smooth out the edge of the rectangle, and often it is enough to make a passable swap.

If it's still not good enough, try the -S switch to do a seamless convert. Be warned, this will take a lot longer and sometimes leads to other glitches.

Crypt1cBunny commented 6 years ago

Thanks for that info! Can this also be done on GPU to speed up the process?

torzdf commented 6 years ago

Convert doesn't appear to utilize the GPU. It takes all the memory, but then doesn't tax my GPU (GTX 1080) at all, unlike extract and train.

Convert also appears to max out just 1 cpu core, so I'm fairly convinced there's room for optimization here, but it's an area of the code I'm a little more hazy on, and I've hit a brick wall each time I've tried to improve speeds on it.

Crypt1cBunny commented 6 years ago

Could the rectangle overlap also be the cause because I don't utilize the alignment json file? So if I have Jimmy Hendrickx to replace obama in a video, I would need to utilize the obama alignment json file (from extraction) in the conversion process?

bryanlyon commented 6 years ago

This was a bug. It was fixed with https://github.com/deepfakes/faceswap/commit/fc457b69aa9b1dc5e97df1bebef1027935f2258a

I apologize for the error. It was due to misloading/saving the weights. Please update to the latest version and revert to a backup of the model and it should work fine.