search

SamsungLabs / rome

Realistic mesh-based avatars. ECCV 2022
Other
428 stars 41 forks source link

Jitter problem. #8

Closed zhanglonghao1992 closed 1 year ago

zhanglonghao1992 commented 2 years ago

Hi～ When I tested it on videos, I found that there was obvious inter frame jitter in the result, as shown below.

https://user-images.githubusercontent.com/17874285/182549247-392a5e21-068a-4a59-8d24-13ebe19ccc50.mp4

Why does it cause such obvious jitter? Is there any way to optimize it.

khakhulin commented 2 years ago

Do you use DECA camera or a camera from the mobile-net?

zhanglonghao1992 commented 2 years ago

@khakhulin I use the default settings in infer.py.

khakhulin commented 2 years ago

So, it should be ok, with the natural jitter from the deca (see video on the site).

nlml commented 1 year ago

@khakhulin Is there a way to reduce this jitter? What is the 'camera from the mobile-net' that you refer to?

khakhulin commented 1 year ago

@nlml I just suggest using the original camera from the DECA, not the distiller version. I think if the jitter is a real problem. then you need to optimize the sequence of the keypoints itself. Since original DECA doesn't guarantee temporal consistency if you run it with per-frame expression transfer.