search

prominenceai / deepstream-services-library

A shared library of on-demand DeepStream Pipeline Services for Python and C/C++
MIT License
271 stars 64 forks source link

support face alignment before recog? #1006

Open HappyKerry opened 1 year ago

HappyKerry commented 1 year ago

After get keypoints , do face alignment before recog ,how to? thanks

rjhowell44 commented 1 year ago

@HappyKerry my apologies for the slow response. Can you please expand on your question? I have not done anything with face recognition. My work and experience are more on the gstreamer / deepstream deployment side.

And sorry, I am not familiar with "After get keypoints" ... are you looking for a model to do face alignment?

Again, more information would be useful and appreciated.

HappyKerry commented 1 year ago

@HappyKerry my apologies for the slow response. Can you please expand on your question? I have not done anything with face recognition. My work and experience are more on the gstreamer / deepstream deployment side.

And sorry, I am not familiar with "After get keypoints" ... are you looking for a model to do face alignment?

Again, more information would be useful and appreciated.

Many thans for your respose, face alignment is some image translation operations, like rotate the face to a target angle. I can implement them using OpenCV packages (like cv2.warpAffine) or other packages. My question here is that, how can I put such operations into deepstream pipeline, and make the pipeline to be: face detect → landmark detect → get face images landmarks to alignment by using opencv → put the processed images into pipeline again → face recognition

rjhowell44 commented 1 year ago

@HappyKerry I would first check to see if there is something available on the NVIDIA container site . If not, you may need to create your own custom gstreamer plugin, see the Plugin Writer's Guide.

The other thing you could consider would be to use two Pipelines, one with an app-sink, the second with an app-source. With the app-sink, clients can receive buffers from a pipeline... and with the app-source, clients can push buffers to a pipeline.

So, in your client callback code, you would receive buffers from the first Pipeline, do the face alignment on the raw buffers, then push the updated buffers to the second Pipeline to complete the process. This might not be the most efficient, but would be the easiest way to get started. Once developed, you could then consider moving the code into a plugin.

Let me know if this sound interesting and I can get you some more info to get started.

YoungjaeDev commented 1 year ago

It's related issue by mentioned me (#1029