Explore mobile

[abrichr]

How can we implement OpenAdapt on iOS/Android?

[0dm]

iOS has Shortcuts that allow for easy & simple automation. I'm not sure if it's worth exploring mobile automation at this time though, but it is a good idea to consider.

Pros

• Shortcuts on iOS guarantee compatibility between other devices, it's very streamlined.
• Could make simple ones that carry the OpenAdapt name for exposure
• Certain actions are very easy to execute, like opening an app and typing
• Can execute javascript within Safari to fill out forms

Cons

• Not sure where to even begin when it comes to the AI aspect, we'd have to code a lot from scratch specifically for iOS
• iOS is extremely locked-down, it'd be difficult to collect recording data and we wouldn't be able to playback like we currently do.
• Much lower demand for mobile automation

MacOS also has these shortcuts, maybe there's a good way to develop them on MacOS and ship to iOS.

[0dm]

I have no idea about Android as I've been trapped in the Apple ecosystem for too long. If anyone has ideas, feel free to add :)

[abrichr]

Related: https://rustdesk.com/ https://github.com/h-mdm/remote-control-android

What does it look like to show desktop screenshots on mobile, and convert mobile user input into desktop actions?

[abrichr]

Closing until this becomes high priority