Detect Streamer's Faces Automatically

[chand1012]

I made a concept script for this exact thing that seems to work exceptionally well while using a simple HAAR Cascade-based OpenCV Python script.

The approximate location for the streamer's face would be based on the auto-generated thumbnails to decrease IO to Twitch. Most of those functions should be copy+pasted into a Lambda that will return the location of the streamer's face for when the auto-generate lambda runs and creates short-form mobile clips.

[gatesyp]

this looks like a good v1 to me

curious to see how it performs in production, there are some caching/efficiency opportunities i can think of

cool feature! :)

[chand1012]

We could definitely cache it in the future, however it should be fast enough for an initial prototype. Using Python's async library I can process about 30 images a second on my system. It would be slower on lambda but it will be fast enough for the first prototype. In the future I can multithread it properly with step function maps.

[chand1012]

Things to note:

• When trying to detect a streamer's face on a thumbnail generated by Twitch, it is best to upscale the image to 2x resolution (from 272p to 544p) as it makes facial detection significantly more accurate.
• Facial detection on lambda is very, very slow. A job that took half a second on my system took well over 15 seconds on lambda (though that could have been due to memory limitations). Going to use parallel processing via a SFN Map on the final version.

[chand1012]

Here is my quick-and-dirty chart of how the lambda would work. There would be one lambda process per thumbnail.