OthersideAI / self-operating-computer

A framework to enable multimodal models to operate a computer.
https://www.hyperwriteai.com/self-operating-computer
MIT License
8.34k stars 1.11k forks source link

Issue using non-QWERTY keyboards #109

Open fpapleux opened 7 months ago

fpapleux commented 7 months ago

I just noticed something interesting. I use a French keyboard ("AZERTY") and when the system "searches", it opens Sppotlight and writes "Google Chro,e" as if typing on a US keyboard, instead of "Google Chrome". On a French keyboard, the "," is located physically where the "m" is on a US keyboard. Any idea how to fix this?

joshbickett commented 7 months ago

Hi @fpapleux thanks for opening this issue. I am not aware of how to fix this. If someone could fix it and open a keyboard compatibility PR that'd be great.

Kreijstal commented 7 months ago

you'd have to tell chatgpt to figure the keyboard layout on it's own, maybe try all keys, make a mental map of it, and then rewire the key-strokes

yilmaznaslan commented 6 months ago

I have experienced a similar issue. I actually already mentioned in my prompt that the keyboard is German-Standard keyboard but it ended up typing faulty characters

1) How to reproduce

Prompt: Go to https://otv.verwalt-berlin.de/ams/TerminBuchen?lang=en. But don't forget that keyboard you are using it as german-standard keyboard

2) Result Since the URL is wrongly typed in ends up as a search result

image

Kreijstal commented 6 months ago

I have experienced a similar issue. I actually already mentioned in my prompt that the keyboard is German-Standard keyboard but it ended up typing faulty characters

1. How to reproduce

Prompt: Go to https://otv.verwalt-berlin.de/ams/TerminBuchen?lang=en. But don't forget that keyboard you are using it as german-standard keyboard

2. Result
   Since the URL is wrongly typed in ends up as a search result

image

there are other scripts for getting a termin in berlin, but I recommend https://wiki.imacros.net/iMacros_for_Firefox but honestly it won't do much, berlin has no termin, until they do.

augustin-grv commented 5 months ago

Same issue here !