Embeddings Not Matching

[sbmalik]

I am extracting embeddings using the image-data and their corresponding embeddings, but when I compare these embeddings they are very similar to each other even if faces are different

[zhouyuchong]

@sbmalik did you do face alignment before extracting? If not, please follow readme to do alignment and try again. :-)

[sbmalik]

Okay sure, I'll let you know after testing

[sbmalik]

Should I have to change nvinfer to something else for main.pyto use your custom plugin?

[zhouyuchong]

@sbmalik nope, just compile the custom plugin and make install it. It is modified on the top of nvinfer, so it is basically the same as nvinfer. make install will replace nvidia official nvinfer.

[sbmalik]

I tried but it is not working, I'll check further and let you know

[sbmalik]

@zhouyuchong it's still not working, I've to change NvDsMetaList * l_user = object_meta->frame_user_meta_list; to NvDsMetaList * l_user = object_meta->obj_user_meta_list; to compile custom plugin in gstnvinfer.cpp:2223 as the error was object_meta has no member 'frame_user_meta_list'. But still not working

all the embeddings still output distance/similarity with very minor difference

[zhouyuchong]

@sbmalik Hi, I've add some new codes in 'custom-nvinfer' to support license plate recognition. So maybe there are some errors. Can you try using retinaface as pgie and remove yolov5? My current pipeline looks like this: retinaface->tracker->arcface, I've tested locally and it works fine:

root@huanxu:/opt/nvidia/deepstream/deepstream-6.1/sources/python_apps/kbds/test# python3 cal_sim.py 
[[0.04977836]]
[[0.22057416]]
[[-0.04537255]]
[[0.35562442]]
[[0.01244287]]
[[1.]]
[[0.05409411]]
[[0.25204449]]
[[0.06085228]]
[[0.26436458]]

Or you can wait some time, I'm still working on this. I'll get you noticed when done. Sorry for all these matters, I have to say it is a tough job since I'm working on my own. :crying_cat_face:

[sbmalik]

okay I'll try on it, can you please confirm if make is still working on custom plugin. without error of object_meta has no member frame_user_meta_list

[sbmalik]

@zhouyuchong I checked after removing the yolov5 and it's working a lot better now...

[zhouyuchong]

@zhouyuchong I checked after removing the yolov5 and it's working a lot better now...

glad to hear that. because the *custom-gst-nvinfer' now can do alignment correctly if the landmarks are generated by a pgie, where lmks are in frame meta. There are some problems if lmks in object_meta, I'm trying to find out the reason.

[sbmalik]

@zhouyuchong Thank you for your help, kindly tell me one thing, for face verification which distance metric you are using?

[zhouyuchong]

I'm using the same method as tensorrtx, here

[sbmalik]

Thank you for solving this issue. Kindly let me know when you resolve issue for lmrk in sgie...

[manhdoan291]

hey shifu @zhouyuchong i make file sucessfully and then i have libnvdsgst_infer.so. How to use gst-infer-custom of you for face-align on deepstream-face-recognition ? I run file main.py , i get a problem: Unknown or legacy key specified 'alignment' for group [property] Unknown or legacy key specified 'user-meta' for group [property]

[zhouyuchong]

after compile it, use

make install

to replace the old one. However, I strongly recommend to have a backup for original gst-nvinfer.