search

ORB-HD / deface

Video anonymization by face detection
MIT License
684 stars 95 forks source link

select faces to not be blurred #51

Open reeseherber opened 1 year ago

reeseherber commented 1 year ago

I was looking into this project and was wondering if it would be possible to select specific faces out of the file to leave unblurred.

StealUrKill commented 1 year ago

I second this

mdraw commented 1 year ago

The face detection model used internally by deface doesn't support face recognition (which would be required for matching a specific face), only general face detection. This feature would be nice to have but I'm afraid implementing it properly would require quite some work and would make the code much more complicated. If I do a rewrite some day, I'll keep this use case in mind.

mthebaud commented 10 months ago

Hi, I have quite the same need, but I have another idea for implementation ? Could it be possible to define a "detection box" that, with a given pixel rectangle, would ignore the outter of this rectangle. My use case : two person facing the camera, I want to blur the right one > I define a global rectangle on the right part and the face of the left one is not blurred. Thanks

mdraw commented 10 months ago

Hi @mthebaud, this feature might be a bit too specific for the main project but feel free to implement this in a fork by filtering the detections dets in this loop by their coordinate range: https://github.com/ORB-HD/deface/blob/1e6a87fbf7dd92c16d36418ed019ea8781ca1d72/deface/deface.py#L83-L89

As you see, detection boxes are already being clipped to the valid frame coordinate range, so you could just change the first arguments of the max() and min() calls respectively.

mthebaud commented 10 months ago

That's great, thank for the information !

nemesis567 commented 8 months ago

Honestly that is hugely specific. Using face recognition seems to be the way to go to achieve this. Just run face recognition and then contour recognition, place the person in a buffer then paste that on top of the blurred frame.

mthebaud commented 8 months ago

Yes, I understand that is specific, so I suggest that that this issue can be closed as we have answers to our questions.

andyg2 commented 8 months ago

I think it would be less specific if the faces were ordered from left to right and the indexes of that list could be an option. Obviously someone might need to cut up a video into segments when that order doesn't change - but for interviews etc. having an option to select one or more faces to skip is a useful option.

StealUrKill commented 2 months ago

I have enabled/added face recognition to it. Works well but it is slow, about 3fps (give or take on hardware). I think I will try and refactor some settings to speed it up. It is using dlib and their models. It is uploaded in my RC branch.

StealUrKill commented 2 months ago

Commenting on this again as I was able to finally get the newest

CUDA toolkit 12.6 cuDNN 9.4.0 TensorRT 10.4 GA for x86_64

Setup on my gpu and successfully building and installing the dlib pip module from source with cuda support.

pip list is as follows

anonfaces 1.0.6rc0 dlib 19.24.99 onnx 1.16.2 onnxruntime-gpu 1.19.2 ----- installed from onnxruntime documention for cuda 12 pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

only here for testing due to the correct lib imports in centerface now. openvino 2024.1.0 openvino-telemetry 2024.1.0

installed but not needed from my testing tensorrt 10.4.0 tensorrt-cu12 10.4.0 tensorrt-cu12_bindings 10.4.0 tensorrt-cu12_libs 10.4.0

Now the face recognition runs at a nice 15fps with a RTX2070

Screenshot 2024-09-11 000637

sfxworks commented 1 week ago

Hey, any update on the above? I was looking at this project and https://github.com/ORB-HD/deface/compare/master...StealUrKill:anonfaces:master but couldn't quite find where to use this feature.

StealUrKill commented 1 week ago

Hey, any update on the above? I was looking at this project and https://github.com/ORB-HD/deface/compare/master...StealUrKill:anonfaces:master but couldn't quite find where to use this feature.

It's under the RC branch and not master. I have a few things I want to finish/fix before switching it over to the master. Specifically instructions for the dlib python wheel.

sfxworks commented 1 week ago

Interesting to test. What is it bound by? During scenes with no faces it's fine, but scenes with faces go from 40 it/s to 2 on a Radeon 6800XT.

Either way this is awesome.

StealUrKill commented 5 days ago

Interesting to test. What is it bound by? During scenes with no faces it's fine, but scenes with faces go from 40 it/s to 2 on a Radeon 6800XT.

Either way this is awesome.

Interesting results. That is far lower than mine as all my machines have Nvidia and using Nvidia cuda with dlib increases the speed 7x usually. Check the readme under the prebuilts folder in the python311 for all my testing results with specs of each of my machines. Have not fully got around for testing the python312 yet. But I feel like it will be the same.

I think the 2 it/s is CPU bound for dlib on actual faces for your machine. I can explain more if needed. I also think there is a better way or possibly a better face recognition that is faster other than dlib. DLIB seemed to be the most straightforward option when searching.

Can you show and explain what you're using in terms of specs and software.

Like are you using dml for onnxruntime? CPU? OPENVINO? Python version?

Also which dlib are you using? Self compiled via python, visual studio, and cmake?

Also what is the size and dimensions of your video that is being used as testing?