search

leezer3 / OpenBVE

OpenBVE- A free train simulator
http://www.openbve-project.net
278 stars 52 forks source link

accessibility for the visually impaired of open BVE #639

Open marrie opened 3 years ago

marrie commented 3 years ago

Description

I am a visually impaired user of open BVE. I and others who use this want to see a bit more accessibility of the app such as speed reading and ways to get to the station. trhe program is 90 percent there as I can use it, but I would love to see better intagration with my screen reader

Reproduction

If the issue occurs in multiple routes/ trains, please provide one or two samples. In order to reproduce the issue and debug it, it's helpful to have the following:

Route

Please provide a link to where the route may be downloaded.

Train

Please provide a link to the train may be downloaded.

Logs

Logs may be accessed through the 'Report Problem' button on the bottom left of the screen. Please provide a log, and a crash-log if the game actually crashes.

Related information

I would like to start a dialogue with you guys on accessibility as the sim is almost there, and since this is open source, I'd be willing to test (not compile) builds for you guys. openBVE Bug Report24_05_2021.zip

leezer3 commented 3 years ago

Do you have any specific examples of the sort of thing you are after / issues you're seeing?

I'm sure something can be worked out, but it's actually quite hard figuring out exactly what you need. This may actually be quite personal too, not sure:) For example, looking at the linked post would suggest to me something like an audible alarm for station stop points might help?

marrie commented 3 years ago

Thanks. Since this is our first time using a train sim for the sighted, I really don’t know what we could do. Let me do a short clip of what we might need to start off with and go from there. Maybe a sound indicating how far we are from the station apart from the announcement we get from the train if it comes with them. Of corse nothing too obtrusive.

As I cannot upload this 140 mb video I have provided a dropbox link https://www.dropbox.com/s/3xabtehbtn0sr45/open%20bve%20demo%20for%20developer.mp4?dl=1 . I hope that hyper linked worked over github as I’m using email to send this.

Thanks and I’m willing to try new things and discuss with my partner some ideas as well. None of us can code by the way, sorry about that.

Thanks again.

leezer3 commented 3 years ago

It's no trouble.

My initial thought would be that we would want to perhaps add a new option for things like this, if we can agree on something that makes reasonable sense. Perhaps an accessibility option that toggles everything on by default?

Existing features

Distance to next station overlay This can be toggled between off, miles and km via CTRL + D & also shows whether the next station is a stop or a pass. It may change a little fast for your screen reader at the minute (constantly updates to 2 decimal places), but that can probably be tweaked.

Speed overlay This can be toggled between off, mph and km/h via CTRL+V Again, this may change a little fast for the screen reader, but see if it's any good for your purposes.

marrie commented 3 years ago

Good start.

  1. The menu when you hit escape has to read by default thereby making it easier for us to switch options and do what I did with speech.
  2. The distances to the station could be communicated via keybind and sound, this would be useful for a sighted person let’s say not looking at the screen as well as a visually impaired/blind train simmer.
  3. The speeds and restrictions could also be bound to a key bind and auto read or read with presses of the key depending on speed. I’m not sure on the latter, and I might be spitting into the wind here.

I agree on the option to turn all of these on, then have the user customize as needed. Is that a good start?

Thanks.

leezer3 commented 3 years ago

The menu may be a little tricky, but I'll see what I can do. We're directly drawing it, without going through the standard Windows libraries, hence why it's not being recognised by default.

A keybind to speak the next station and speed / restriction is also a good idea, although judging by the speech speed of your screen reader, I'd say that it probably would need to be separate for visually impaired / otherwise. Not sure if there's a suitable multi-platform TTS library handy either, will have to look into that.

Other things: From your video, it appears you're having problems selecting trains sometimes. This is because a train is also a folder (and so technically can contain sub-trains) I'm sure we can handle this a little better, and possibly stop entering the folder if there are no sub-trains present.

I won't make any promises that any of this will be quick or easy to implement, but we'll definitely see what we can do to make things better.

(for my reference) NVDA API: https://github.com/nvaccess/nvda/blob/master/extras/controllerClient TOLK multi-screen reader library (?) https://github.com/dkager/tolk

marrie commented 3 years ago

I did slow it down so you could hear it, but we could speed it up as well. TTS might also be a good idea, or an option to toggle between the two. Do you think the menu could be spoken by tts as well, maybe if the screen reader won’t be able to do so?

marrie commented 3 years ago

to add to my last comment about the menu, I meant the tts and or screen reader could read the options as we arrow through them, this includes the sub menus as well, if at all possible. Like a standard menu bar.

thanks.

jessieppp commented 3 years ago

i agree with everything that @marrie has said so far. i am also another blind train simmer. love the sim so far like she said it is almost there.

leezer3 commented 3 years ago

ScreenReaderTest.zip

Please try this test build. I've added screen reader support to the in-game menu, which appears to me to work OK with NVDA, although obviously I haven't tried to use this in real-life.

jessieppp commented 3 years ago

ok i will test this when i get home how do i do this do i just unzip this and overrite this over the exsisting directory?

leezer3 commented 3 years ago

Yes, that's right.

Assuming this works OK in real-life, it looks like it'll be reasonably easy to add screen reader support to most bits, we just need to agree on what's needed :)

jessieppp commented 3 years ago

So far from what little I tested the menu when you hit the alt key works I was even able to read the jump to station dialogue. I’m going to do some more testing when I get home to see what else can be done I think the only thing that would be needed now would be a way to find out how far the station is from you and things of that nature but looks like we can start implementing these things little by little and testing them I will wait to see what Sarah has to say has to say as well.

Sent from my iPhone

On May 25, 2021, at 06:32, Christopher Lees @.***> wrote:

 Yes, that's right.

Assuming this works OK in real-life, it looks like it'll be reasonably easy to add screen reader support to most bits, we just need to agree on what's needed :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

marrie commented 3 years ago

I got a network error, however when checking my download folder I somehow got two copies of the thing. I’ll test and submit a report shortly.

Thanks so much.

marrie commented 3 years ago

Ok, in doing some testing, it works, I have a question though, or maybe a bug report. Does it say that this is the main menu in print? If it does not, disregard, if it does, indeed say main menu or an indication that this is the main menu, my screen reader is not indicating as such. I don’t think you will need the clip, however here it is https://www.dropbox.com/s/gtvownvaq4tdlop/BVETest01.mp4?dl=1 just in case. This is kind of a think out loud.

Thanks.

leezer3 commented 3 years ago

No, it doesn't do that.

Logical addition though.

marrie commented 3 years ago

Do you think that should be added in print as well? In case let’s say the sighted person’s elbow touches the escape key and they wonder why the train won’t move? Lol! I don’t know if that would look good aesthetically, but that is a suggestion anyway.

Thanks

leezer3 commented 3 years ago

ScreenReaderTest2.zip

Updated so that a bunch more stuff works with the screen reader.

With regards to having the menu in print: It's an overlay over the screen, and (at least from a sighted person's perspective) it should be pretty obvious; You can't see much else!

jessieppp commented 3 years ago

things are working so far so good in what little testing i did. now i would need something to help me aline with the station unless there is something i am missing.

leezer3 commented 3 years ago

Not yet :) I think that (and current speed, limit etc.) will likely have to be a keypress combo, but still need to think about / implement that.

jessieppp commented 3 years ago

Hopefully people won't get mad if I overrun the station. As a blind user it's really hilarious when I try to use the trackside camera to try to align the sound of the train with the station but I haven't yet been very successful at that.

jessieppp commented 3 years ago

Or is there a way well I am pulling into the station where the screen reader with Tommy arrived at such and such station and then I'll know when to stop since I'll be going slow enough?

marrie commented 3 years ago

OK. I will figure out those keyboard shortcuts later and test. Thank you so much for all the work that you were doing.

marrie commented 3 years ago

I’m not sure if this is a GitHub issue, but for what ever reason when I click the link to download my browser tries to download the file twice; upon hitting escape after the second download when the first one is successful it throws a network error. I can still get the test fine though.

I will look and submit a report later today.

Thanks.

marrie commented 3 years ago

I love the direction this is going.

Here is the drop box link to the think out loud.

leezer3 commented 3 years ago

Errors: These are pretty much harmless, and can be ignored, especially in your case. You're missing at least one of the visual NYCTA object packs (for the track by the looks of it)

Stop position: On the list of things I'm thinking about. We have an existing visual overlay, but this obviously doesn't translate to screen readers etc. at the minute.

SI units: Everything internally runs on these, sorry. Some of the speed overlays can be converted to mph etc. but the overrun and stuff which I think you're referring to are only in M

marrie commented 3 years ago

Ah Ok. So, who ever built the train missed something, I guess? Don’t worry about the meters and SI units. The program I’m basing my ideas off, (there is a blind train sim that is kind of but not as good as this one) is in meters and KM. so I’m used to it. Lol. No big deal. Don’t worry about converting. We have our googles for that. Lol! Any ideas on how we could make the overlays accessible to us, as that might be a bug bare. I’m up for the challenge though testing wise. I can’t code. Heha.

Thanks.

leezer3 commented 3 years ago

ScreenReaderTest3.zip

Another small update. This adds a tone when the station stop overlay is active. It'll start out pretty slow, and should be a near enough constant tone when you're at the stop target. However, this one will need enabling via the Options first: On the first page, in the Verbosity checkbox, tick Accessibility Aids

Next job on my list is to get it to announce via screen reader when speed limits change. I'm inclined to keep this behind the same option (as sighted players don't get this either, and need to rely on signs / route knowledge)

It'd also be interesting to know whether screen reader users are likely to have a spectrum of vision ability, and thus want different amounts of things announced, or whether we should assume that if a screen reader is active, everything should be announced audibly?

jessieppp commented 3 years ago

how do we inable the stop over lay?

jessieppp commented 3 years ago

after i rote the comment i figured out how to do it and it is working ok have made some good stops for now.

marrie commented 3 years ago

I love the idea., however for those of us like myself with pitch sensitive issues, I would like to see instead a beep that starts out short and gets faster the closer you get to the station. Upon entering the station to the platform, the beep should then be constant, and if you pass the station the beep should either stop, or get slower, I’m not sure about that yet. I do like the idea though, and it is a good start.

Thanks.

marrie commented 3 years ago

I really don’t know on that one, I guess it would be up to us just like the sighted person to depend on signage I hope I understand that correctly. If not, please please let me know as I’m new at this train sim thing.

Some of us who use screen readers do have some vision as well, do you think down the road customizability in how much and how little is spoken might be an option? I’m thinking far future when we have everything ready. I’d rather right now have spammy than not enough. We can always adjust things later.

marrie commented 3 years ago

I've got to head to work but I got to

  1. test the AI and
  2. test the platform sound.

I like the AI and the message I get from NVDA. Very very good job on this.

Hopefully my idea on the station sound is a bit clearer now that I voiced it. Sorry about the rush but I had to voice them now before I forgot. Basically it is what I said in a prior comment. We're in London this time.

here is the file

jessieppp commented 3 years ago

oops for some reason my screan reader has gone bonkers it is now reading all the posts for the speed tring to hit c t r l v and not working for turning off the speed over lay.

marrie commented 3 years ago

I will get this, to, if I am going way too fast. Is this the signage you were talking about? Or is this something different.

Thanks so much.

cblackstone1977 commented 3 years ago

Hey all. So, when driving a rout in open BVE I notice that occasionally I hear a notification that says that I have passed a stop signal. What I was thinking of is a chime when you are approaching a stop signal. Also the platform sound is a little ear piercing when the tone raises in pitch. Here is how I suggest we solve this one. So, the tone goes like this tone then a pause and then the tone goes to more of a continuous tone as you approach the platform.

marrie commented 3 years ago

Exactly what I was thinking, at least about the tone. I kind of demo everything there is to show in the progress we are making in this video @.***:9/OPENBVE-d02:8> for the public and talk a grate deal about the tone, as right now that is a bit of a show stopper, not by much. I even show how I use the cameras to my advantage in combination with the beep. Feedback is welcome on what I might have missed in this tour.

leezer3 commented 3 years ago

Try this: ScreenReaderTest4.zip

I've modified the station stop point sound to use volume not pitch. (speed without playing with pitch is a much bigger problem) It also now announces / shows on screen new speed limits with the accessibility aids option turned on.

Will look into signals next. My initial thought is that this needs a two-pronged approach:

  1. Announce when within 500m (?) of a signal.
  2. Add keybind to show onscreen / announce signal aspect.
marrie commented 3 years ago

I agree with the two-pronged approach. 500 meters might be fine. That sounds good. And the key bind might work as well. I will be playing with this for a few minutes before work.

Thanks so much.

marrie commented 3 years ago

I want you guys to test it, I can get used to it, but it seems like this is less forgiving, at least for me. I seem to be missing the station and the volume drops too quickly. I like the premise behind it though and the idea makes sense, at least it’s worth a try. Btw I’m not going too fast when I hear the beep. Distance though next would help this way we know weather to go full power or not. I’m not sure yet. I want thoughts from other testers on here.

Thanks so much.

jessieppp commented 3 years ago

you can be a little bit off after the volume of the beep but not by much.

marrie commented 3 years ago

I noticed that as well. It seems to actually drop a bit too quickly. I have to crawl. I would show that off, but I misplaced the vid.

Thanks so much.

marrie commented 3 years ago

My thoughts so far.

I like the premis behind it though. If we were to do pitch maybe 13 half steps to make an octive would be better instead. they would be short beeps. To see an example in an audio game, allthough it's not a very good one compared to open BVE, visit my podcast page and search for dreamy train. Something like that, a faster beep or a short beep until you get to the platform. Here is the video I just made with all of this summerised and shown.

Note, the translation of dreamy train is horrible but that's because it is due to a translator.

thanks, you rock.

cblackstone1977 commented 3 years ago

I have noticed one announcement is missing when you have the speed overlay turned on it doesn't tell which setting you have it setto. Atleast NVDA doesn't announce whether the speed overlay is of set to KMH or MPH. I realy like the reading of the speed postings. They are notto spammy. I was also thinking of a series of 3 short beeps that wouldsound once you are getting close to a station. When approaching a signal 2 slightly longer beeps wold be a good idea when you are getting 500 meeters from the signal.

marrie commented 3 years ago

I kind of like that better than my idea, then the solid beep meaning the platform is there.

You got the speed overlay to work? How? And btw it is in kph I believe.

jessieppp commented 3 years ago

yes chris how did you get it to work.

jessieppp commented 3 years ago

The only thing that we are really missing, is how far we are from the station, also, the speed at which we are moving.

leezer3 commented 3 years ago

ScreenReaderTest5.zip

Some more work on this. Not done anything about the stop beep yet. Station announcements aren't perfect either, as it announces any station, whether you're due to stop there or not.

Again, feedback please on whether this is approximately what you're after :) At some stage soon, I'll add this to the main auto-generated daily builds, and see about adding a small accessibility page on the main site too.

Changes:

Keybinds: These are not set by default on existing installations. (as setting them might conflict with existing keybinds) To get them, you need to either hit the Reset to defaults button in the customise controls screen, or add them as a new control.

These are as follows: ACCESSIBILITY_CURRENT_SPEED - Announces the current speed. Set to CTRL + SHIFT + S by default. ACCESSIBILITY_NEXT_SIGNAL - Announces the next signal distance / aspect. Set to CTRL + SHIFT + A by default. ACCESSIBILITY_NEXT_STATION - Announces the next station. Set to CTRL + SHIFT + T by defualt.

marrie commented 3 years ago

So far so good. I need to play with it a bit more to get used to it. I don’t know if this can be coded, but is there a way the thing can say the name of the station? I can then look in my route and see if that is part of it this way, I know weather to stop or not. I notice my driving a bit smoother now.

Keep up the good work, and thanks a lot.

marrie commented 3 years ago

I love the direction it's going. I think though an offset is being read wrong as we will see for example 14kph and we will according to the signs be going at 80 or higher. I think I did demo this in the think out loud, quite by accident.

Keep up the good work, can the speed thing be looked at though as I don't want wrong results.

thanks so much

jessieppp commented 3 years ago

i also saw that as well i was just now doing a track in new york and when i was going down a hill it was saying sign post indicates 40 kph and i was doing about 70 and when i looked it was saying 15 kph i think.