VoiceOver support - Githubissues

tbodt commented 5 years ago

Right now, VoiceOver support is just what we get automatically from having the terminal be rendered as a webpage. Here are some of my observations from trying to use the app with VoiceOver on:

It's possible to scroll the WebView slightly in VoiceOver mode.
The bug where the terminal doesn't resize when the keyboard appears initially is worse with VoiceOver, since it's harder to realize you're missing part of the screen, and very non-obvious that the way to get it back is to select the terminal and double tap.
The slick little arrow button is entirely non-functional with VoiceOver. And it's the only way to press arrows.
The control, tab, and escape buttons are read as "control", "tab", and "escape", which is fortunate. The "hide keyboard" button is read as "black down pointed triangle", which is less fortunate.

tbodt commented 5 years ago

I've heard from a VoiceOver user who said it would be nice to automatically read new terminal output. I'm not sure about the best way to implement that though.

codeofdusk commented 5 years ago

That's me! Thanks for taking time to do this.

Yes, auto-reading output would be wonderful. Any particular reason you chose to use webKit instead of native iOS controls? I think the latter would make this easier to implement, and probably have better performance overall, etc.

tbodt commented 5 years ago

When I looked into it, the only existing open-source iOS-compatible terminal emulators that I could find were web-based. My goal was to get something working quickly, so I just used a WebKit view. Porting Alacritty was also something I considered, but that takes time and effort 😛

yatil commented 5 years ago

Use a (hidden) div with aria-live=“assertive” and copy the output into it once you receive it. VoiceOver should pick it up automatically.

mcb2003 commented 5 years ago

I spent some time fiddling around and came to the following conclusions of what improvements could be made to the accessibility of iSH:

[ ] Typed characters are not spoken by voiceover. This, I assume, is because voiceover will only echo characters back to you if you're typing in a text field, which you're not in iSH.
[ ] New output is not spoken by the screen reader, as it would be with orca on linux or indeed voiceover on the mac. However, I personally feel this is fine: it's really easy to just touch the last line to have it read out. As @yatil pointed out, using some Aria attributes could make this possible.
[ ] Arrow key functionality is not accessible. Me and @tbodt discuss this on the discord server:

I wonder if I could have a directional swipe with the arrow button focused do an arrow key and add a description explaining how to do that

Sounds good to me. You could also use voiceover actions, but that would be cumbersome for multiple key presses.

codeofdusk commented 5 years ago

Re auto-reading output, this is important especially on mobile. VO actions for arrows could work, as could the directional swipe.

Bill

On 6 Dec 2018, at 23:59, Michael Connor Buchan notifications@github.com wrote:

I spent some time fiddling around and came to the following conclusions of what improvements could be made to the accessibility of iSH:

Typed characters are not spoken by voiceover. This, I assume, is because voiceover will only echo characters back to you if you're typing in a text field, which you're not in iSH. New output is not spoken by the screen reader, as it would be with orca on linux or indeed voiceover on the mac. However, I personally feel this is fine: it's really easy to just touch the last line to have it read out. As @yatil https://github.com/yatil pointed out, using some Aria attributes could make this possible. Arrow key functionality is not accessible. Me and @tbodt https://github.com/tbodt discuss this on the discord server: I wonder if I could have a directional swipe with the arrow button focused do an arrow key and add a description explaining how to do that

Sounds good to me. You could also use voiceover actions, but that would be cumbersome for multiple key presses.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tbodt/ish/issues/101#issuecomment-445074304, or mute the thread https://github.com/notifications/unsubscribe-auth/ACXISyKnxTqcuCmLd6qJolfKMQhT7QGTks5u2a9KgaJpZM4Ym78Z.

alekssamos commented 5 years ago

does not work edit mode (cut, copy, paste). copying and pasting text is difficult. please fix.

Rotate the VoiceOver rotor to either "words" or "lines." To rotate the VoiceOver rotor place two fingers on the screen and turn them left or right. If you select "lines" you will select one lines at a time and if you select "words" you will select one word at a time.
Rotate the rotor to "edit" mode and then swipe up or down to hear all of the options. It will give you options such as cut, copy, paste, define and more depending on which app you are using. bdmtech.blogspot.com/2012/02/how-to-cut-copy-and-paste-while-using.html

codeofdusk commented 5 years ago

Any updates on this? Particularly for character echo and autoread.

tbodt commented 5 years ago

I'm afraid not, sorry.

tbodt commented 4 years ago

I just got auto read to work, which makes character echo automatically work. I'll push an update with this today.

tbodt commented 4 years ago

Question for the room: I just discovered if you double tap and hold on the arrow keys, it passes the events through to the button, which means you can just drag to send arrow keys. I think this is better than the current setup where the control is adjustable and you double tap to toggle between sending left/right and up/down. Is it so much better that I should remove the code for making it adjustable and change the hint text to tell you to double tap and drag?

codeofdusk commented 4 years ago

Even though it’s slow, the most efficient relative to the other options would probably be to add voiceover actions to the terminal for sending the arrow/modifier keys.

Sent from my iPhone

On Oct 15, 2019, at 17:09, Michael Connor Buchan notifications@github.com wrote:

Yes you should just do that.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

danestange commented 4 years ago

hello. As most of you probably know, ish is now one of the most accessible terminal's for the iOS platform. I have noticed, however, that a couple things do not work. For example, when flicking up or down on the arrow slider, it only reads a couple things. For example, if i'm in a nano window, i can not edit a file, as it says, space, for example. Let's say i wanted to edit /etc/motd, i type nano /etc/motd, it opens, but i can not edit anything in said document, as the arrow's don't read like they should. Maybe there's already a fix, idk. If so, please let me know. I have been using ish ever since me and the developer talked and more accessibility was added, and i love it so far.

62f commented 3 years ago

Informational: I have an iPhone 4s on which voiceover set to only read aloud words when I misspell something and it gets autocorrected. I cannot imagine the filibuster that would be created on this 7s+ in iSH with nearly every word being challenged. Also: after much procrastination, I finally updated iOS to 13.7 this morning and iSH now displays copy/paste after a select operation while the keyboard is hidden. Time will tell if webview behaves better and stops playing keep-away with the touch kb.

lawrenceper commented 1 year ago

VoiceOver doesn't work well on latest iSH app and iOS 16.1. iSH autohides the keyboard when trying to navigate to listen to terminal output. I have to use Braille Screen Input to make the keyboard stay consistently, which is OK. But when using explore by touch, if I move my finger outside of the virtual keyboard, then it autohides itself. Bluetooth external keyboards no longer seem to work. Can you please fix?

saagarjha commented 1 year ago

@lawrenceper can you try iSH 1.3 and see if it helps?

ish-app / ish

VoiceOver support #101