Deepstream

[hirwa145]

Is there a way i can deploy using Nvidia Deepstream ? Or create Deepstream app from this?

[shubham-shahh]

check this, dlib model and facenet have similar output https://forums.developer.nvidia.com/t/face-recognition-using-dlib-with-jetson-nano/167172

[hirwa145]

So it is possible

[nwesem]

@hirwa145 this was asked before and i think it is a great idea. Let me know if you are interested to work on that

[shubham-shahh]

@nwesem I am working on a similar project, but first I'm trying to use dlib's model for face recognition with deepstream

[hirwa145]

Have you succeeded with it?

[shubham-shahh]

currently i am working on https://github.com/riotu-lab/deepstream-facenet this, because to run this repo with deepstream will require us to configure mtcnn as with the deepstream.

[shubham-shahh]

Hey, I managed to get facenet working with deepstream but it's a python implementation. shall I put up a pull request? @nwesem

[hirwa145]

@shubham-shahh yes please. Can you upload it. I am very interested to see how you managed to make it work with Deepstream

[shubham-shahh]

I have a python edition, currently I am working on a C++ version as well, shall I PR a python version?

[hirwa145]

Yes, you can PR the python version

[hirwa145]

@shubham-shahh you pull requested the python version?

[shubham-shahh]

not yet, @nwesem should create a new branch for the python version, meanwhile, I'll commit the project to my account.

[hirwa145]

Yeah. That is also okay, you can commit it to your account whilest we are waiting for him to open new branch

[shubham-shahh]

Yeah. That is also okay, you can commit it to your account whilest we are waiting for him to open new branch

I'll complete it by this weekend

[hirwa145]

Alright, thank you

[nwesem]

hey @shubham-shahh, cool! Thanks for contributing to this project! I am looking forward to see your implementation. I added you as a collaborator on the project, please add your code to a branch called python-develop and I will check it out and merge it if everything works as expected. Currently I am pretty busy with work, but I will take the time to test your implementation asap. If you can't create a new branch just push it to your forked repo and send me a pull request

[shubham-shahh]

hey @shubham-shahh, cool! Thanks for contributing to this project! I am looking forward to see your implementation. I added you as a collaborator on the project, please add your code to a branch called python-develop and I will check it out and merge it if everything works as expected. Currently I am pretty busy with work, but I will take the time to test your implementation asap.

Hey, I hope you are doing fine, thanks for the branch I'll submit my work by this weekend.

[shubham-shahh]

Hey, @nwesem I updated the develop branch of my fork with the python implementation, @hirwa145 please test it and let me know if there are any issues, and once this is tested ill put up a pull request on your develop branch and it would be great if we keep them in separate branches. Here

[hirwa145]

Hello @shubham-shahh . Great job doing deepstream implementation. So far i tried the python implementation and it is working perfect and thank you. I am currently testing the cpp implementation version even tho it is taking long for testing the sample app and i am not sure why. But i have a question, is there a way to implement the recognition where faces in video files are compared to saved images in the folder and display the name of the person instead of '"person". Thank you.

[shubham-shahh]

Hello @shubham-shahh . Great job doing deepstream implementation. So far i tried the python implementation and it is working perfect and thank you. I am currently testing the cpp implementation version even tho it is taking long for testing the sample app and i am not sure why. But i have a question, is there a way to implement the recognition where faces in video files are compared to saved images in the folder and display the name of the person instead of '"person". Thank you.

Hello, did you follow the guide step by step? was there any issue at any step? and thanks for testing. and about your question, I will implement that functionality this weekend if possible but if you want to implement it by yourself i can tell you the approach which you can follow. thanks again

[hirwa145]

for python implementation, everything works as expected. But for C++ implementation, i have to first test the test sample, so it seems it stuck here NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 . D you know why?

[shubham-shahh]

for python implementation, everything works as expected. But for C++ implementation, i have to first test the test sample, so it seems it stuck here NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261 . D you know why?

Is this the sample App? or the one i included in the repo? whats your video source .mp4 or .h264?

[hirwa145]

sample_720p.mp4, as mentioned from your instructions

[shubham-shahh]

sample_720p.mp4, as mentioned from your instructions

and is it the sample app present already or you are using the repo app?

[hirwa145]

yes, available since i installed deepstream. it must be tested on that sample? Or i can test on another .mp4 file?

[shubham-shahh]

yes, available since i installed deepstream. it must be tested on that sample? Or i can test on another .mp4 file?

The one that is already presemt works with .h264 files only and the one in the repo works with mp4 files and rtsp streams

[hirwa145]

So which means when i am doing testing c++ implementation. i do this ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.h264 instead of ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4

[shubham-shahh]

So which means when i am doing testing c++ implementation. i do this ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.h264 instead of ./deepstream-infer-tensor-meta-app -t infer /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_720p.mp4

right

[hirwa145]

Perfect. Now i am able to run C++ implementation successfully and detecting faces as it should. Sometimes misses face that are out of focus.( probably due to resolution/quality of the video) , also if i noticed that when testing on .mp4 or longer video, there is no quiting option, so i have to wait to wait for video to finish so that i quit the app.

[hirwa145]

Also if it is possible, could you show me how i can implement the function that the name of face owner is displayed please? Thank you

[shubham-shahh]

Actually the quit option is like you need to ctrl C on the terminal, you need to keeo terminal visible as the sink is overlay sink, if you want really accurate performance for face detection you can change yolo face's precision to FP32 but it will really take a hit on performance which is not worth it. What's your approach to implement it? C++ or python?

[hirwa145]

I would like to test both(yolo face's precision changed to FP32) and compare.

I prefer C++ more, because the input is .mp4 file, also does it accept 1080p resolution?

[shubham-shahh]

Any type of resolution will work, you just need to change, the resolution in the app and in case you are planning to go with C++, you need to first save the embeddings in a file and then, you need to match them with the current embeddings received

[hirwa145]

you need to first save the embeddings in a file and then, you need to match them with the current embeddings received How do i do this please? Will need to retrain the model?

[shubham-shahh]

No no, it works on the similar principal like the master branch of this repo, can you gimme time till this weekend I'll try to implement it and let you know

[hirwa145]

Sure, no problem. Take your time. And also what about changing yolo face's precision to FP32?

[shubham-shahh]

In dstest pgie config file change the network mode

[hirwa145]

I set network-mode=1 to network-mode=0 for FP32

[shubham-shahh]

I set network-mode=1 to network-mode=0 for FP32

Yes correct

[hirwa145]

Is there a way to know how many fps? For testing purpose.

[shubham-shahh]

Is there a way to know how many fps? For testing purpose.

There's a inbuilt implementation to know fps in deepstream, but in that case you need to refer a test app that has demonstrated its use.

[hirwa145]

Did you manage to implement the face recognition features where faces are compared from saved faces to determine who face is?

[shubham-shahh]

Did you manage to implement the face recognition features where faces are compared from saved faces to determine who face is?

Not yet as I'm occupied, but ill complete it by tomorrow if possible.

[hirwa145]

from issue #25 , is it possible to implement with this deepstream app?

[shubham-shahh]

from issue #25 , is it possible to implement with this deepstream app?

Yes, I'm working on it, for both the branches

[hirwa145]

From test_facenet_trt.py , i saw that you made these changesinput_file_path = 'pat/to/your/testimage.jpg' dataset_embeddings_path = '/path/to/your/dataset' How dataset is prepared?

[shubham-shahh]

From test_facenet_trt.py , i saw that you made these changesinput_file_path = 'pat/to/your/testimage.jpg' dataset_embeddings_path = '/path/to/your/dataset' How dataset is prepared?

Hi, I am working on that. You need to extract embeggings of each person, from images and save it in a pickle file.

[hirwa145]

Hello @shubham-shahh, how far with the implementation?

[hirwa145]

Are embedding saved as.npz file or should saved as .pkl file? Currently i have all photos needed to be recognized from video file. One labeled folder has photos of same person.

Does util.py need to be in same folder as dataset when being executed?

[shubham-shahh]

Are embedding saved as.npz file or should saved as .pkl file? Currently i have all photos needed to be recognized from video file. One labeled folder has photos of same person.

Does util.py need to be in same folder as dataset when being executed?

npz or pkl both can be used