search

godbout / Wooshy.docs

it's like Alfred but for the UI
https://wooshy.app
218 stars 0 forks source link

Elements that don't have visible text labels #150

Open alex35mil opened 3 months ago

alex35mil commented 3 months ago

It's quite challenging dealing with icons and other elements that don't have visible text labels.

Suggestion: detect if an element doesn't have a visible text label, and if it's the case - add a label above the element with either accessibility text attached to the element or randomly generated characters if the element doesn't have anything.

godbout commented 3 months ago

first think to make sure if case you don't know, you can search by Element types and more and use The Inspector to see what terms would highlight the Element.

https://github.com/godbout/Wooshy.docs/assets/121373/8bf5b9ff-f18e-4def-8a54-0c741db4ff18

that being said, more in the next comment.

godbout commented 3 months ago

It's quite challenging dealing with icons and other elements that don't have visible text labels.

Suggestion: detect if an element doesn't have a visible text label, and if it's the case - add a label above the element with either accessibility text attached to the element or randomly generated characters if the element doesn't have anything.

is this more of an issue with webpages and/or Electron apps? the current issue is that Wooshy does some magic to wrap all the data and make it available through The Input. because the web and Electron apps are the Wild Wild West. they don't respect anything. so you can't check whether the AX "Title" is empty, because maybe what they're showing is actually the "Help", or "Description", or some other random field they've decided to use. the only thing we would be able to know for certain is if they haven't filled any data and everything is empty. if they've filled at least one field, we can't be sure whether it's the one shown or not on the browser.

but i haven't dug more in a while. personally or 1) i'm use to the page and i know what to look for 2) or it's a new page and i just go with what i think would work and most of the time it does lol

so lemme know specifically where you're having those issues so that i can think of a better solution than the current one. thanks.

alex35mil commented 3 months ago

It's the web most of the time. Sometimes icons don't have any attributes, sometimes it's hard to guess the text, and sometimes there are many repetitive controls, like in long lists, so it would be much easier to have a unique random label to jump to the control I need rather than tab my way through all of them.

godbout commented 3 months ago

It's the web most of the time. Sometimes icons don't have any attributes, sometimes it's hard to guess the text, and sometimes there are many repetitive controls, like in long lists, so it would be much easier to have a unique random label to jump to the control I need rather than tab my way through all of them.

yeah, i'm having the issue currently with a site that doesn't fill its AX attribute properly, so i have to go through the list of Targets:

https://github.com/godbout/Wooshy.docs/assets/121373/6cefcc35-032c-4e44-ac89-4d895b74ab94

but you don't need to press any key several times. if you hold it then Wooshy speeds through the Targets. that works well with me because i'd still rather type something, and use j/k to go through the Targets, coz i'm a Vim and habits junkie.

if this is not good enough for you tho, seems you may prefer using Labels through other apps like Shortcat, Vimac or Homerow: https://github.com/godbout/Wooshy.docs?tab=readme-ov-file#alternatives-to-wooshy

there's requests for a Label mode in Wooshy, and i've reopened the relevant issue, but the current chance to have this happen is like not even 10%: https://github.com/godbout/Wooshy.docs/issues/146