The mouth is terrible, and side view

[I8Robot]

There is my video, is the head too big? After training, the result is terrible. Train command:

python main.py data/{dir_name} --workspace model/trial_{dir_name} -O --iters 60000 --asr_model ave
python main.py data/{dir_name} --workspace model/trial_{dir_name} -O --iters 100000 --finetune_lips --patch_size 64 --asr_model ave

There is my result:

[thatnerdyaigirl]

I am also having this issue how to reolve?

[I8Robot]

I am also having this issue how to reolve?

not yet

[varunjain99]

I also had a blurry mouth in one training video... In another video, I used more diverse expressions and the mouth came out clearer. My hypothesis is that if the training video doesn't have diverse/extreme enough expressions the NERF is unable to accurately figure out how the face looks. Do you have any other ideas?

I also have an issue where for some training videos, the entire head seems to shake and jitter... have you seen this?

[I8Robot]

I also have an issue where for some training videos, the entire head seems to shake and jitter... have you seen this?

Yes, I have the same problem, and don't have any solution. Maybe need to smooth the preprocess result, but I haven't tried yet。

[varunjain99]

@I8Robot @thatnerdyaigirl any insight into the lip blurriness / mouth artifacts?

One issue I discovered might be that the expressions influence the lips, and if the expressions haven't really been seen in training the lips can be blurry. To alleviate, the ambient attention loss can be increased.

But I still get somewhat blurry lips - any learnings from your experimentation?