Use facemesh for filters?

[tjasmin111]

Interesting project. Is it possible to use facemesh to change face and make nose smaller? How can I do that?

[Lirui991225]

describe more clearly

[tjasmin111]

I mean, let's say I want to make some changes in the face landmarks like those beauty apps. For instance, to make nose smaller or lips bigger and such. How to achieve that?

[Lirui991225]

Too easy！

[tjasmin111]

Hmm can you please explain how? We can "get" the facemesh but how to use that to change the face landmarks?

[Lirui991225]

My English is poor,do you know chinese?Can we communicate in Chinese？

[Lirui991225]

or we can use the wchat

[tjasmin111]

我不懂中文。 但你可以用中文解释，我可以使用谷歌翻译。 所以我的问题是，我们可以“获取”面部网格，但如何使用它来改变面部标志？

[Lirui991225]

Perhaps you can perform some transformations on the pixels within a certain range around the landemarkes.

[Lirui991225]

Why can you replay to me so quickly

[tjasmin111]

Hehe because this is important to me. I need to know how to do it.

[Lirui991225]

Are you working or studying

[tjasmin111]

What kind of transformation? Can you write an example that gets facemesh and transforms nose?

[tjasmin111]

I am an intern.

[Lirui991225]

Oh, where are you from? Which school are you from

[Lirui991225]

me，too

[tjasmin111]

Ok! I'm in the US. California.

[Lirui991225]

OK!What is your major，my main focus now is on computer vision and AI

[Lirui991225]

What kind of transformation? Can you write an example that gets facemesh and transforms nose Can't your third stars solve your problem

[tjasmin111]

Yes I'm a computer scientist focusing on computer vision. Do you think you can write a sample app to see how we can change landmark? What transformation should we use??

[Lirui991225]

OK!I'll try to write it down.

[tjasmin111]

Ok thanks. If we can do it for nose and make it smaller, then I can do it for all landmarks.

[Lirui991225]

What time is it in your area

[Lirui991225]

i think i can use the opencv todo it

[tjasmin111]

It's 2:30am! And still awake, googling for this problem. I hope you can do it with opencv. So after knowing the mesh coordinates, you use opencv for transformation?

[Lirui991225]

Some mesh deformation algorithms can be used, such as Laplacian deformation.

[Lirui991225]

I need to know your specific task now.

[tjasmin111]

Let's make nose smaller. That's the task.

[tjasmin111]

How to use laplacian transformation? Should we pass the nose coordinates or the whole face mesh?

[Lirui991225]

Let's make nose smaller. That's the task. The image or 3Dfile？

[tjasmin111]

Prob image? Let's assume we stream from laptop camera.

[Lirui991225]

Prob image? Let's assume we stream from laptop camera.

Image stream of camera？

[tjasmin111]

Yeah, because imagine a person is trying to slim nose on his phone camera. Actually, it is 3D mesh of face bc we use facemesh.

[tjasmin111]

Can you just make a small puthon code that make our nose look smaller?

[Lirui991225]

import cv2 img = cv2.imred('./test.jpg') nose_mesh = [] new_width,new_height=a,b#a and b are the new width and height of the nose letf,right,top,bottom=nose_mesh[:,0].min(),nose_mesh[:,0].max(),nose_mesh[:,1].min(),nose_mesh[:,1].max() nose_np_array = img[top:bottom,left:right,:] new_nose_array = cv2.resize((new_width,new_height))

do a img fusion

[tjasmin111]

Did you test it? Can you please post How does the before and after image look?

[Lirui991225]

use the 'Poisson Image Editing'

[Lirui991225]

I didn't test it, just provided my ideas

[tjasmin111]

Hmm you just get the area of nose and resize it. But how to stitch it with the face with no artifact? Please test it. I don't think it's that simple

[Lirui991225]

use some image fusion algorithms

[Lirui991225]

Just like 'Poisson Image Editing'.

[tjasmin111]

Hmm poisson image editing fills the empty pixels with background image...yeah worth a try

[Lirui991225]

Yes，you can have a try，there are some tutorials on Google that you can learn

[Lirui991225]

May I ask how much your internship salary is ?

[Lirui991225]

Hmm

[tjasmin111]

Poisson algorithm needs a mask..not sure how to generate it. Internship is not much...40$/hr

[Lirui991225]

It‘s so cool,mysalary is just10000￥ per month.

[tjasmin111]

Country to country is different. If you come to US, you can get $50

[Lirui991225]

That's true

[Lirui991225]

Poisson algorithm needs a mask..not sure how to generate it. Internship is not much...40$/hr

The mask can be got by the nose area of mesh

[tjasmin111]

But it shouldn't be a single mask. We have to run it continuously for camera. Face pose can change

The mask can be got by the nose area of mesh