search

joshfarrant / shortcuts-js

A JavaScript iOS 12 Shortcuts creator
https://shortcuts.fun
GNU General Public License v3.0
1.69k stars 102 forks source link

Getting Shortcuts to "tap" items on the screen #687

Closed SuePB59 closed 2 years ago

SuePB59 commented 2 years ago

I need to create shortcuts that will interact with elements on the screen of an app. For example if there is a menu icon I'd want the shortcut to "tap" that and then to "tap" a specific item on the menu. Is there no way to identify a location on the screen to touch? This can be done with the Accessibility>Voice Control>Show Grid if you are giving voice commands, so why isn't there a way to script it into a shortcut?

joshfarrant commented 2 years ago

The short answer is that if you can do it with the Shortcuts app you can (in theory) do it with Shortcuts JS. If you can't, then it's a no-go I'm afraid.