normal-computing / fuji-web

Fuji is an AI agent that lives in your browser's sidepanel. You can now get tasks done online with a single command!
Apache License 2.0
193 stars 13 forks source link

Handle typing when no input element is focused (expedia fake input) #112

Open mondaychen opened 2 months ago

mondaychen commented 2 months ago

There are cases when the agent decides to type in an "input" on a page, but it turns out to be a fake one. For example, on expedia, this "Date" label looks very much like an input image

However, no input is focused after clicking it. Instead, it opens up a dialog expecting button clicking.

image

Typing in this case is tricky because it can trigger keyboard shortcut websites, or at least "space" would cause browser to scroll to next screen in Chrome.

We need to think of a way to handle this problem