Any demo to inference with mobile screenshot with prompt?

cooelf / Auto-GUI

Official implementation for "You Only Look at Screens: Multimodal Chain-of-Action Agents" (Findings of ACL 2024)

https://arxiv.org/abs/2309.11436

Apache License 2.0

174 stars 15 forks source link

Closed truebit closed 9 months ago

truebit commented 10 months ago

Thanks for the nice work. Is there any demo code to inference a prompt with a mobile sceenshot to get the response from Auto-UI?

truebit commented 9 months ago

finished after reading model.py and utils_data.py and fetch_features.py

kirtishrinkhala commented 8 months ago

Can you please share how you did this?