search

georgeretsi / smirk

Official Pytorch Implementation of SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis (CVPR 2024)
https://georgeretsi.github.io/smirk/
MIT License
126 stars 10 forks source link

Adding a Tracker to SMIRK #2

Closed raincrash closed 2 months ago

raincrash commented 3 months ago

Since the pre-training is based on MICA, Is there a way to use Metrical_Tracker that comes with MICA, with SMIRK?

If there is no direct way and need a bit of development effort, I am happy to do it and send a PR. Just need directions on how to go about it.

filby89 commented 3 months ago

Hey, thanks for your interest in SMIRK ! If I'm not mistaken in https://github.com/Zielon/metrical-tracker/ MICA predictions are used to regularize shape throughout the iterative fitting. The most straightforward way to incorporate SMIRK here then would be to regularize w.r.t. SMIRK expression params as well.

Nonetheless, I think that the SMIRK framework could probably work very nicely in an iterative fitting process with the cycle path as well, although we did not have the time to fully explore that yet.

Feel free to check both options and let us know on the results :) However, I think this is not in the scope of this repo and would probably better exist in its own repository.

raincrash commented 2 months ago

Thank you for the response! I will close the issue. Will also try to work on this, if something interesting comes up, I will post here for completeness.