search

bbonik / facial-landmark-frontalization

Function to frontalize non-frontal 2D facial landmarks generated from the DLIB library
MIT License
21 stars 10 forks source link

Convert frontalize landmarks back to original #6

Open peterwisu opened 1 year ago

peterwisu commented 1 year ago

Hi, I wonder if there is a way to convert the frontalized landmark back to its original pose? Thank you

bbonik commented 1 year ago

Hi. Can you describe why you need to do this? What is the scenario you want to implement? The typical use-case is that you start from the non-frontal landmarks and you frontalize them. Therefore, you still have the original landmarks. You don't need to reconvert the frontalized ones, because you already have the actual original ones.

peterwisu commented 1 year ago

@bbonik Thank you for your reply. I am trying to make a machine-learning model that can generate a new lip landmark according to speech. So, I have used your facial landmark frontal technique to convert all facial landmarks I detect from the dataset to frontal form and fed lip landmarks to the model. And this works well very well with your frontal technique well done!! But the problem with my model right now is that during the inference, the lip landmark that generates is in frontal form. I want to transform this generated lip back to the original landmark pose so that its pose does match others. So what I can do to solve this is un frontal the frontal landmark along with its new lip(generated) so that the new lip has the same pose as the original but with a different expression from the original lip. I hope this won't sound confusing.

skyler14 commented 1 year ago

@peterwisu did you make any progress on the reverse frontalization? @bbonik what changes would you say need to be made in order to port this into other face landmark libraries btw? @peterwisu on your lip gen project did you end up using it and if so did you use dlib or port to something else?