Add navigation assistance for the visually impaired

[StephenCWills]

Discord user trtr1000 has suggested that navigation assistance will be necessary for visually impaired players. I'm not sure yet what specifically will be involved; this is just to document the lack of navigation assistance.

[AJenbo]

There should be a sound effect when you cross doors, arches and stair cases to indicate there presence. The effect should use reverse distance (there is a video that will better explain how this works, I'll try and find it).

The sound effect would play when you are crossing one of the mentioned "objects" on a straight line.

Here is the video that shows a game build buy blind users for blind users: https://youtu.be/AcfMLycJYpo?t=68

There is a "checkpoint" (blue circle) that works the same way that stairs should.

The norm is to have that sound play instead of your foot steps and then have the sound adjusted to the objects position rather then yours.

But thinking about it I don't think we actually need to disable foot steps and can just have the "object" emit the sound.

When I say "object" instead of object, it's because there are no arch or stair objects. It's more like specific tiles. So we might need to introduce a new entity for this or figure something else out.

[ryanmusante]

Would this sound have an effect on how the AI responds? Or is it purely for QOL?

[StephenCWills]

It should have no effect on AI. It's an accessibility feature for people who can't see the screen.

[StephenCWills]

Here are some additional comments from trtr1000, for consideration.

we can do random dungeons, as long as we can find a way to navigate, and get all the info that everyone else gets. it doesn't all have to be sounds, it can be status notifications that the screan reader reads out too.

some of us listen to screan readers really fast. also, I'm not sure if theirs dodging involved, but if their is, could their be a sound cue for that, and do items auto pickup, if not, have the item play a constent sound untill we walk over it, and then have the screen reader read what item it was that we picked up and how many.

[Lirin111]

Hi,

Let me introduce myself a bit. :)

I am totally blind (sightless) drummer, singer, audio engineer, voice actor and accessibility consultant. I am working with game developers and modders to improve accessibility for the blind players.

I was working on accessibility for games like: -Hades - First ever fully playable roguelike game for the blind due to the mods created: https://caniplaythat.com/2021/05/17/hades-accessibility-mods-for-blind-visually-impaired-players/

-Slay The Spire - First fully accessible roguelike deck-building game thanks to it's modding system (the devs rejected working on the accessibility so We, blind gamers decided to create the mods): https://bradjrenshaw.github.io/say-the-spire

Lastly, I was working with French developer COWCATGames on BROK: The InvestiGator - First and only one fully blind accessible point-and-click adventure game with beat em up elements: https://www.brokgame.com/accessibility.php

I've discussed some of the accessibility aspects with @AJenbo

The accessibility needs sounds for: -Walls - Great for navigating dungeons and city to hear some kind of wind sounds playing on your left / right and when the wall is on the north / south the wall sound could play with pitch up / down effect. Doors / stairs - Stationary sounds you will be able to hear when close enough. This sound should be positioned in stereo related to the placer indicating if you need go to the left / right. When closer to the object the sound should be louder. Containers / breakables / items - No idea what to do about that as there are a lot of stuff around many times and adding audio to all of these could overload the game with tons of sounds. The game could have some kind of tracking feature IE. Use the D-Pad left / right to select categories like items, containers etc and up / down to select the item from the category. After clicking a key on it the sound of that could play or the character could get there automatically (similar to how you are interacting with enemies around so you can press attack and yoou will track the enemy close to you)

Just my initial ideas! :)

[yuripourre]

I am so glad that this game could be played by even more people.

I would like to add my two cents. Diablo is a real-time game and very often you are surrounded by enemies. I would like to elaborate a little after reading the other comments.

Audio Feedback

I suggest we use OpenAL (or similar API) with support to real 3d sound: For fixed elements (walls, chests, bookshelves, NPCs) we could have continuous 3d sound (we would probably need to calculate the distance from the closest walls).

For movable elements (enemies and other players) we could have the sound triggered when they move or other state changes (just like the first example @AJenbo posted).

In regards to screen reader support I would leave that to items, inventory, etc. Oh, and monster HP. I am afraid that if everything is readable there will be too many elements at the same time on the screen.

Interactions

I think we could start with something similar to the gamepad but for keyboard. I didn't think too much about this but wanted to mention.

[AJenbo]

Last night talking to trtr1000 we also came up with the idea of calculating the path for targeted entities but not actually walking, instead highlight it with audio cues.

[SightlessKombat]

By way of an introduction, I'm also an accessibility consultant, gamer without sight and content creator, as well as currently being Accessible Gaming Officer for RNIB (Royal National Institute of Blind People) in the UK.

I've been credited for work on Sea Of Thieves, God Of War Ragnarok and Horizon Forbidden West, as well as working on games including Gears 5 (for Navigation Ping: Escape Mode), just to name a few.

I've played numerous hours of Diablo 4 in its various stages and I think this could be a great project, but just a couple of points to note: The screen reader element would only be when you pick up items, use inventory etc. Audio cues could indicate the rarity of an item (Legendary, common etc) as long as they're distinct enough.

In terms of an actual navigation route, though I'm unfamiliar with how the original game works, if there are linear paths to your next or current objective, just have that rendered as an audio cue to follow.

I'm happy to provide assistance where I can including with testing and the like as this is developed and I'm really excited what this could hold for the future of retro accessibility. Keep up the great work!

[AJenbo]

Diablo doesn't exactly have rarity, but it has: normal, magic, and unique.

This is one of the main tasks for version 1.6.0. We are currently preparing to release 1.5.0. We'll keep this issue up to date as things progress.

[DakkJaniels]

In terms of an actual navigation route, though I'm unfamiliar with how the original game works, if there are linear paths to your next or current objective, just have that rendered as an audio cue to follow.

Levels are randomized, so the entrance, exit, any quest based locations will not be in the same location from game to game. While there is a mini-map, it only shows you areas you've already explored, so there is no "next objective" feedback like you'd have in Diablo 4. The only thing that remains the same from game to game is the Town, and certain quest location set pieces (I think).

[heinermann]

Hello, I wasn't fully aware of the issue/discussion but I wrote this on Discord

This is for blind people? That sounds interesting. I think just any kind of ambient sounds and action feedback could help with the direction of things. Flowing water from the streams, sounds from the tavern, blacksmith, fountain, cows, NPCs, portals, etc. Footsteps could sound different based on how close to a wall or door you are. As far as items go, a shuffling noise specific to the item if something walks over it could be neat. We could use one of those AI voice generators to get the NPCs to sigh or make sounds in their respective voices. All that and improving the directional audio a bit. I'll hop back on this later because I really like the idea.

I really think these kinds of ambient sounds and improved audio can make the game feel more immersive generally in addition to helping blind gamers.

Some further ideas: Different fountain sound than mentioned above when it's polluted (plus occasional coughing NPCs as an indicator for that quest, except for Pepin, Adria, and Cain because lore or whatever). Different footsteps for different surfaces (i.e. caves vs cathedral, to help determine when you have changed zones), or whether you are walking on dirt or grass in town. Ambient sounds for town trees, torches, and cave lava also. Swinging weapon at a wall to get the sound of the wall. Enemy movement sounds. A crow at the entrance to the cathedral could guide players there from afar.

There's really a lot that could go into this.

[heinermann]

I'm not sure what the gotos are for royalty-free sfx. Freesound, and any others?

NPC Mumbling

Here are some vanilla NPC sounds I generated with Elevenlabs. Cain seems to be particularly difficult to work with. I searched for onomatopoeia but didn't find anything workable (people claim that the word is the onomatopoeia but it is not what I want). Things like "ah", "hmm", "humm", "oh", "whew", "phew", "eh" seemed pretty straightforward. Sneezing, drunk sounds, sighing, and wounded townsman sounds were particularly difficult to generate.

DiabloBlindNPCs.zip

There's a Cow1.wav and Cow2.wav in the game. I didn't do anything with the special monsters (Gharbad, Snotspill, Zhar, Lachdanan) or Hellfire.

Stepping on Equipment

Just trying to come up with a list of items that would have different sounds when stepped on. Thoughts? Is there an item database somewhere so I can know exactly which quest items there are?

Equipment

• Axe
• (Wooden) Club
• Mace/Hammer
• Flail
• One-handed Sword
• Two-handed Sword
• Bow
• Staff
• Small Shield
• Large Shield
• Light armour
• Medium armour
• Heavy armour
• Small shields
• Large shields
• Cap
• Other helms
• Ring
• Amulet

Other

• Potion/Elixir/Oil
• Rune (Hellfire)
• Book
• Scroll
• Gold (something like this)
• Ear
• Various Quest Items (unique sfx for most)

Interactables

Things like shrines can have a humm, fountains could have water sounds, stairs could have wind gusts.

How about other objects such as bookshelves, barrels, doors, lecterns, chests, sarcophaguses, etc.?

Open Questions

• How will infravision work? My thought is it will reveal sfx through walls but it will be difficult to convey that there is a wall in the way, maybe some kind of muffling?
• How will "the blind" work? My thought is don't play movement SFX while they are hidden.
• Spells and other elements (such as acid from spitters on the ground) will need some thought as well.

Footsteps

• Town: Dirt, Grass
• Cathedral: Stone slabs, wet/slushy stone slabs (bloodied floor in Butcher's chambers)
• Catacombs: Stone
• Caves: Rocky ground
• Hell: Bone ground??? Unsure
• Nest: Something squishy?
• Crypt: Fancy tiles

[AJenbo]

people claim that the word is the onomatopoeia but it is not what I want

interjections

[heinermann]

Just want to note that in vanilla:

• There is a footstep sound from the player but it is the same everywhere, I'm not sure if it comes from other players too (probably not).
• There is no feedback for picking up an item off the ground.
• None of the monsters seem to make any sound unless they attack you.
• There are also random rubble piles and blood pools on the ground.
• Monster corpses too.
• Caps have the same sfx as other helms.
• Wooden clubs have the same sfx as maces and swords.

An idea for picking up an item, a sound combination indicating what was picked plus where it went (belt vs inventory).

[AJenbo]

The text to speech engine would resolve issues like knowing what item you are highlighting before you chose to pick it up.

[StephenCWills]

There is no feedback for picking up an item off the ground.

FYI... https://github.com/diasurgical/devilutionX/blob/2624625431d482d4c443aa8a854d93e25a9248c4/Source/options.h#L461-L462