search

USTC3DV / FlashAvatar-code

[CVPR 2024] The official repo for FlashAvatar
MIT License
143 stars 18 forks source link

Flame models #8

Closed alasokolova closed 4 months ago

alasokolova commented 5 months ago

Hey, thanks you for such an amazing work! It really looks great!

I am trying to run your code, but in the model initialization Flame models is needed. Could you upload it or explain where to get it? Which version of Flame do you use? I have found generic_model.pkl and FLAME_masks.pkl, but don't see landmark_embedding.npy.

alasokolova commented 5 months ago

And one more question concerning flame: Are the metrical-tracker outputs obtained by metrical-tracker? Can any other tracker be used for this purpose?

xiangjun-xj commented 4 months ago

And one more question concerning flame: Are the metrical-tracker outputs obtained by metrical-tracker? Can any other tracker be used for this purpose?

Yes, we use MICA and metrical-tracker and version of Flame is the same as them. Also, we have uploaded the file for you. Of course, better tracker leads to better results. Curious about the tracker you'd like to use.

alasokolova commented 4 months ago

Thank you for the answer! I have not tried any other trackers yet, but see that some trackers estimate flame partially (not all 300 shape parameters and 100 expression parameters, for example). May such trackers be applied for Flash Avatar?

I tried to run metrical-tracker, but it seems to be quite slow relative to FlashAvatar, so all the processing together turns out to be not so fast (as flash avatar itself). Am I right? Or have i done something wrong?

Anyway, the results of FlashAvatar are absolutely impressive!

xiangjun-xj commented 4 months ago
  1. We attach Gaussians to mesh surface initially and you can try different trackers or even mesh models.
  2. Yes, you are right. Accurate and fast tracking is another great challenge.
Tiandishihua commented 4 months ago

Thank you for the answer! I have not tried any other trackers yet, but see that some trackers estimate flame partially (not all 300 shape parameters and 100 expression parameters, for example). May such trackers be applied for Flash Avatar?

I tried to run metrical-tracker, but it seems to be quite slow relative to FlashAvatar, so all the processing together turns out to be not so fast (as flash avatar itself). Am I right? Or have i done something wrong?

Anyway, the results of FlashAvatar are absolutely impressive!

In metrical-tracker, lmk_path = imagepath.replace('images', 'kpt').replace('.png', '.npy').replace('.jpg', '.npy') is kpt7.npy ? lmk_path_dense = imagepath.replace('images', 'kpt_dense').replace('.png', '.npy').replace('.jpg', '.npy') is kpt68.npy?