search

AvdLee / RocketSimApp

Enhancing the iOS simulator for productivity
566 stars 20 forks source link

Simulate Fingers Covering Screen #502

Open ThatGuySam opened 2 months ago

ThatGuySam commented 2 months ago

Is your feature request related to a problem? Please describe. I haven't been able to find any tool that simulates fingers covering the screen for interaction so that me and my team know exactly what the user will see when the app is physically in hand and can factor finger position into design decisions.

Also, this would be very helpful to show designers how their designs are work as a real-life touch experience.

image

Describe the solution you'd like Ideally an option to cover the screen with simulated fingers that follow the cursor in real time with realistic joints.

Alternatively maybe just show the fingers while your cursor is pressed down.

An option to set the hand's opacity, default 100% opacity

An option to switch between various holding positions

Auto-mode that switches between Thumb, Two Hands, etc so that it's clear what's not reachable by a given hand position.

image

Describe alternatives you've considered I was thinking about building a web app to test this on screenshot but it's not quite the same effect as a tool that's always on and realtime.

Additional context Articles I've found on Finger UX

Modern Touch-Friendly Design (2020) https://addyosmani.com/blog/touch-friendly-design/

Hand and Finger Positioning https://www.uxmatters.com/mt/archives/2017/05/design-for-fingers-touch-and-people-part-2.php

How We Hold Our Gadgets(2017) https://alistapart.com/article/how-we-hold-our-gadgets/

AvdLee commented 2 months ago

@ThatGuySam thanks a lot for suggesting this! It's not going to be an easy feature to build and I wonder how many users will use this (managing expectations here). Any resources (e.g. Swift code) that could help me build this would greatly improve likability of me building this 🙏

ThatGuySam commented 2 months ago

Nothing like this even exists and, in my opinion, this is a huge blind spot in Mobile UX so I could certainly see a lot of value for larger companies doing very deep UX improvements.

I'm not a Swift dev but it could be fun to port some existing Hand stuff from VisionOS or Unity/Unreal: https://medium.com/codex/how-to-use-vision-hand-pose-in-swiftui-a24b7a7e5721 https://developer.apple.com/documentation/vision/detecting_hand_poses_with_vision

ThatGuySam commented 2 months ago

You could also just use a Transparent PNG or SVG of a hand and anchor the Finger to the mouse position