search

heestand-xyz / PixelKit

Live Graphics in Swift & Metal
MIT License
885 stars 66 forks source link

Question: Support for CoreML / Vision? #17

Closed f0rr0 closed 3 years ago

f0rr0 commented 4 years ago

I was hacking together a demo where I had to do portrait segmentation (Zoom like background removal) without chroma key. I decided to use Deeplabv3 and found this project: https://github.com/jsharp83/MetalCamera which has all the parts properly setup. With a few minor changes I was able to macOS compatible as well.

This would make a great addition to PixelKit.

My use case is: remove the background from realtime video and merge it with a custom image/video.

Relevant File: https://github.com/jsharp83/MetalCamera/blob/master/MetalCamera/Classes/CoreML/CoreMLClassifierHandler.swift

heestand-xyz commented 3 years ago

Hi, Deeplabv3 is nice, tho it’s hard to get it going in realtime, I was trying it out here: DeepLabPIX. SaliencyPIX is faster tho low res. A blur, a threshold, a channel mix and a blend might do the trick.