search

swiftablellc / jabberwocky-head-tracking-kit-ios

Jabberwocky Head Tracking Kit for iOS
https://www.jabberwockyapp.com
Apache License 2.0
10 stars 0 forks source link

.touchUpInside without blinking + camera live preview #8

Open SwiftyFlow opened 2 years ago

SwiftyFlow commented 2 years ago

Would it be possible to .touchUpInside without blinking? just when hitting the button box...

Also have the camera live preview?

Thanks!

broag commented 2 years ago

Both would be possible. I'll take a look at what it would require in a bit here...

SwiftyFlow commented 2 years ago

wow you answered so fast! If it makes more sense, I'd like to track the nose, and if the nose hits the button, consider it pressed... All that by seeing the user's face so he can see his nose hitting the button... Legend!

broag commented 2 years ago

Yeah I know what you are talking about. It should be pretty easy, you really just want a focus trigger which already exists for highlighting. If you can stand a delay, I think there is already a feature that you could enable to do this by "Dwell Clicking".

broag commented 2 years ago

So Jabberwocky HTKit built using a plugin based architecture and you can create your own HTFeature classes (in your project) and then initialize them. Although this might take a bit of digging in on your part, you can create both of the things you want without any changes to the JabberwockyHTKit.

The Feature loading looks like this:

let features: [HTFeature.Type] = [
        BannerWarningsFeature.self,
        // CursorBlinkClickFeature.self, // -- DON'T NEED THIS
        CursorDrawFeature.self,
        // CursorDwellClickFeature.self, // -- REPLACING THIS WITH YOUR OWN HOVER CLICK FEATURE...
        YourCursorHoverClickFeature.self, // --YOUR HOVER CLICK FEATURE
        FaceMeshFeature.self, // --THIS OR YOUR OWN CAMERA PREVIEW FEATURE
        CursorFocusAndClickAnimationFeature.self,
        CursorFocusFeature.self,
        CursorRecalibrationFeature.self
    ]
HeadTracking.configure(withEngine: ARKitHTEngine.self, withFeatures: features)
HeadTracking.shared.enable()

And you would want to create a Feature that is "very" similar to: https://github.com/swiftablellc/jabberwocky-head-tracking-kit-ios/blob/master/JabberwockyHTKit/Features/CursorDwellClickFeature.swift

And for the camera live preview there is a mesh based feature that already exists for demoing. Maybe it'll work for you: https://github.com/swiftablellc/jabberwocky-head-tracking-kit-ios/blob/master/JabberwockyHTKit/Features/FaceMeshFeature.swift