Anybody else getting washed out colors

tencent-ailab / V-Express

V-Express aims to generate a talking head video under the control of a reference image, an audio, and a sequence of V-Kps images.

2.03k stars 250 forks source link

Anybody else getting washed out colors #42

Open oisilener1982 opened 2 weeks ago

oisilener1982 commented 2 weeks ago

The output quality of v-express is not that great. Sadtalker does not have this problem. I hope tencent would be able to fix this

tiankuan93 commented 2 weeks ago

What kind of script, data did you use? And what kind of results?

oisilener1982 commented 2 weeks ago

I used this tutorial to install locally: https://www.youtube.com/watch?v=ttEOIg9j2B4&t=327s

These are the sample results and the output does not have the same quality as the reference. Capture2 Capture

The video below was created using sadtalker and it retained the skin color and texture. https://youtube.com/shorts/qJR0m4jGeGw?feature=share

This is the Video output from V-express https://youtube.com/shorts/LCOiMrVLwaE?feature=share

tiankuan93 commented 2 weeks ago

Since we only use portrait data for training, we currently only support a large and clear face image. Instead of a bust photo, you should enter an input like the one below.

oisilener1982 commented 2 weeks ago

Since we only use portrait data for training, we currently only support a large and clear face image. Instead of a bust photo, you should enter an input like the one below.

Thanks for the clarification but in the image below the reference image is high quality with clear skin texture while the output video has washed out color and too bright light. The animation is actually good and much better than Sadtalker but v-express has some output quality issues. IF this can be fixed then i will definitely switch to v-express

1111

zhangjun001 commented 2 weeks ago

I did not test the gradio version. The ComfyUI version works well for me.
My settings are shown bellow:

https://github.com/tencent-ailab/V-Express/assets/12435654/cdc40693-ca96-4590-b4bb-0e5efa854d0f

oisilener1982 commented 2 weeks ago

I just installed comfyui but i dont have any idea :)

Hope someone could post a tutorial about how to use v-express in comfyui

oisilener1982 commented 2 weeks ago

I did not test the gradio version. The ComfyUI version works well for me. My settings are shown bellow:

test3.53.mp4

Would it be ok if you try this exact image?

This is my output

https://github.com/tencent-ailab/V-Express/assets/29243313/7303985e-d1f8-4c38-9908-f73144854f43

zhangjun001 commented 2 weeks ago

Here are some results from my test

https://github.com/tencent-ailab/V-Express/assets/12435654/89cad9af-74ff-4450-b155-67b952d7fdce

https://github.com/tencent-ailab/V-Express/assets/12435654/a847f76c-78eb-4aff-a088-cc88d46812f8

https://github.com/tencent-ailab/V-Express/assets/12435654/bf32408c-def7-4980-8ea6-da609500daf3

zhangjun001 commented 2 weeks ago

Take a look at this one. I adjusted the parameter of cfg=1.5.

https://github.com/tencent-ailab/V-Express/assets/12435654/6fdc3898-0beb-48ee-84e1-eb5bbbef2e42

https://github.com/tencent-ailab/V-Express/assets/12435654/417e27be-053d-478b-8f86-e49cb31ec8cd

zhangjun001 commented 2 weeks ago

It's our fault for not giving good instructions and guidance. Maybe after some time, I can provide a simple tutorial to tell you how to select kps videos and how to set parameters.

oisilener1982 commented 2 weeks ago

Thanks for this. Test3.61 and test3.63 both have washed up color. Test3.59 and Test3.68 are fine but a lot can still be improved in terms of quality. What settings do you have for 59 and 68? Is it just the cfg 1.5 what was adjusted? Thanks in advance

I hope you improve the skin color and texture to match better the Reference image.

oisilener1982 commented 2 weeks ago

This is what i mean with better skin texture and output that closely resemble the Reference image. I made this from sadtalker but sadly the emotional expression is missing and v-express is way better in this aspect.

https://github.com/tencent-ailab/V-Express/assets/29243313/adb74adf-daa2-4895-8d29-929bb71590fc

zhangjun001 commented 2 weeks ago

Thanks for this. Test3.61 and test3.63 both have washed up color. Test3.59 and Test3.68 are fine but a lot can still be improved in terms of quality. What settings do you have for 59 and 68? Is it just the cfg 1.5 what was adjusted? Thanks in advance

I hope you improve the skin color and texture to match better the Reference image.

Since I want to do it using a retargeting strategy, the results will be more natural. And because naive-retarget cannot be effective for any facial posture, choosing a kps video with a roughly similar facial posture to drive can often achieve better results.

https://github.com/tencent-ailab/V-Express/assets/12435654/f7ef390d-f7ec-4521-b07e-1f82f8e9d969