Introduce role-based braille mapper

[nvaccessAuto]

Reported by aliminator on 2014-04-30 14:14 Very often in NVDA and especially when using braille, role names and their states shown are either lengthy or at a "wrong" place. By referring to "wrong place" it is meant that it is not intuitive and users have to search e.g. for the control state (e.g. for checkboxes). This is the case even when braille display is tethered to review. Furthermore the display representation is coupled to the settings set to speech. Uncoupling them has the advantage of working faster especially when users work with both braille and speech. A "braille mapper" can give full control to the user to customize braille output; thus every user can define his own scheme. Following can be configured:

• Text representation of control type
• Position of control type or state label relative to the text
• Text might be underlined using dots 7 and 8

Braille mappings can either be global or application-specific and are stored in the nvda.ini configuration or application-specific one. Braille mappings are defined on a per-controltype basis where the controltype is specified as ints which, at the moment is not intiutive. If a mapping is not defined for a controltype it falls back to the usual representation scheme. The mapping can take any text plus specific patterns which indicate variables such as values/names, hotkeys etc. This is a list of patterns:

Pattern Meaning %t Name or value %s states %h keyboardShortcuttext %i current index %I total amount of items %c columnNumber %o rowNumber %d description

Example: To show the state of the checkbox in front of its name or value (without hotkey) following entry has to be in the configuration: 5 = %s %t

Furthermore the key dot78roles in the configuration file lists all roles which text should be marked by dots 7 and 8. Both ruling mechanisms may work together.

A GUI hasn't been created yet. Eventually, only a few users do need this and hence can do it manually (as it was the case with gestures) or an addon can be delivered to provide the GUI. It would be great to have at least this functionality in the core since it can not be provided by addons.

The patch I attached contains modifications of the config/init.py (for the configuration template object) and the braille.py file which contains following modifications:

• Adjustment of getBrailleTextForProperties(**propertyValues):: Resorted calls. If rule does not exist for control type continue with "old" behaviour.
• def mapToRawText(text, role, states, **options): Contains the new mapping code. Currently is quite inefficient and some parts are missing. But I would like to complete it if this functionality would be accepted.
• NVDAObjectRegion.update(self): Added check whether role match config, and if this is the case set flag to true.
• Region.init(self): Added Attribute which is checked at NVDAObjectRegion class.
• Region.update(self): Added routine marking braille cells with dots 7 and 8.

[nvaccessAuto]

Attachment brlmapper.patch added by aliminator on 2014-04-30 14:15 Description:

[nvaccessAuto]

Comment 1 by jteh on 2014-04-30 21:59 Thanks for the suggestion and patch. I haven't taken a look at the code yet, but I have several concerns here:

• If there are changes to the default scheme that would benefit all users, these need to be discussed and applied to the default scheme if appropriate. Saying that it is configurable is not enough if the majority of users want something else. However, any changes need solid justification and feedback from users.
• Any framework for customisation should include the default scheme, not just fall back to it. That is, we shouldn't have two separate presentation functions.
• I don't follow how this framework fixes the issue of braille output being linked to the same settings as speech. As I see it, that needs to be solved another way.
• If we're going to do something like this, it should ideally be a more general framework so it can be used for speech as well. For example, some speech users want states spoken first.
• Can you explain where displaying dots 7 and 8 for a role is useful? I understand this for list item selection, but that requires mapping both a role and specific states. Or do you already support this?
• This is going to be too difficult for the majority of users to configure. I'd be reluctant to accept this without a GUI, since each time we've added configuration without a GUI (e.g. symbol pronunciation or input gestures), this has been a problem for a lot of users. At the very least, it needs to be extremely well documented.

[nvaccessAuto]

Comment 2 by aliminator (in reply to comment 1) on 2014-05-05 10:01 Replying to jteh:

Thanks for the suggestion and patch. I haven't taken a look at the code yet, but I have several concerns here:

• If there are changes to the default scheme that would benefit all users, these need to be discussed and applied to the default scheme if appropriate. Saying that it is configurable is not enough if the majority of users want something else. However, any changes need solid justification and feedback from users.

I am not sure whether all users will agree to the same presentation scheme. If this is the case it should be definitely hard-coded and used as default. But if this is not the case then the it could be assumed that the best representation scheme is the current one. Therefore two representation functions. Even though the scheme agreed on is hard-coded - it is nevertheless a two-step process which in this case is expressed using two functions.

1. When there is no information available about roles to represent use the hard-coded rules.
2. If there is information available use the rules stored in the config.

• Any framework for customisation should include the default scheme, not just fall back to it. That is, we shouldn't have two separate presentation functions.

I understand the need to reduce complexity by using only one representation function. But I consider this functionality as the extension of the existing one. Pre-defined mappings could be hard-coded (see above) or they could be stored on a per-language basis such as the character descriptions. But wouldn't this be too complex?

• I don't follow how this framework fixes the issue of braille output being linked to the same settings as speech. As I see it, that needs to be solved another way.

It does not fix that problem completely but provides a mechanism to "separate" those settings by removing. That means that you can e.g. remove the hotkey from the braille representation although its appearance is specified in the settings and thus is still spoken.

• If we're going to do something like this, it should ideally be a more general framework so it can be used for speech as well. For example, some speech users want states spoken first.

I didn't have a look at the speech code, so cant' say how complex it would be. But if it does work similar to the braille layer it should be possible to generalize the code.

• Can you explain where displaying dots 7 and 8 for a role is useful? I understand this for list item selection, but that requires mapping both a role and specific states. Or do you already support this?

This was intended for clickable elements such as buttons or links. It doesn't work well in virtual buffers - I couldn't figure out the reason. At the moment mapping on a per-role basis is supported as annotated in the code.

• This is going to be too difficult for the majority of users to configure. I'd be reluctant to accept this without a GUI, since each time we've added configuration without a GUI (e.g. symbol pronunciation or input gestures), this has been a problem for a lot of users. At the very least, it needs to be extremely well documented.

I would like to add both GUI and documentation. But this should be done after the basis of this framework are discussed and thisfeature is going to be integrated.

[nvaccessAuto]

Comment 3 by aliminator on 2014-05-05 10:09 The patch attached before, should be a starting point (by example) for this discussion. If this feature is requested I would be glad to further develop it.

[nvaccessAuto]

Comment 4 by halim on 2014-05-16 06:55 first thx aliminator83 for this patch and your great work. Some code optimizations should be made to your code but the implemented functionality is realy useful and works here.

@nvda Developers: Please have a look to this patch. and consider applying it to nvda. It doesn't change nvda's current braille representation if no mappings are defined in the ini file.

[nvaccessAuto]

Comment 5 by winman3000 on 2014-06-05 12:06 I would appreciate it if this feature can be implemented. In Europe, especially in Germany, working with Braille is very important. In Germany, almost all blind person has a Braille display. Displaying Dots 7 and 8 in lists etc. would be also very useful. In this case, you can turn off speech and work only with Braille if you don't have headphones.

[nvaccessAuto]

Comment 6 by werwoelfchen on 2014-06-08 12:37 In the patch there are unfortunately no heading levels any more, perhaps they could be shown after the heading like "Text of the Heading h(1 till 6)". I understand wery goot when you say that the level at the beginning is not necessary but I think the heading level is important to understand the herarchy of a web site using different levels of headings..

[nvaccessAuto]

Comment 7 by aliminator on 2014-06-10 07:22 Open the nvda.ini and find the line beginning with 40. Either delete it or put a # at the beginning and save the file. Make sure the file changed is not overwritten again when the config should be saved automatically.

[nvaccessAuto]

Comment 8 by jteh (in reply to comment 2) on 2014-09-10 10:04 I forgot there were open questions here. See below:

Replying to aliminator:

I am not sure whether all users will agree to the same presentation scheme. If this is the case it should be definitely hard-coded and used as default.

I think the first step, then, is to work out what users actually want here. It's very possible some users would prefer slight deviations, but these are not the primary concern. The bigger question is whether there are major changes desired by most users. Even if there are significant differences of opinion, it may still be better to implement smaller options (e.g. a check box to display states first) rather than a template based system.

The problem with a template based system such as proposed here is that it is extremely difficult to take all possible requirements into account. Furthermore, once it's implemented, it's very hard to support additional requirements later without breaking compatibility with the original scheme. For example, the current patch can't handle the following:

• Dots 7 and 8 for selected list items (role as well as state). This is a request that has been made by several users, so it is particularly relevant.
• A single abbreviation covering multiple conditions; e.g. vlnk for visited link, h1 for heading level 1. These are just examples; there would be more cases it can't cover. My aim isn't to devalue your work at all. The point is that something which appears to be extremely flexible sometimes isn't that flexible or future proof in the longrun.

I understand the need to reduce complexity by using only one representation function. But I consider this functionality as the extension of the existing one. Pre-defined mappings could be hard-coded (see above) or they could be stored on a per-language basis such as the character descriptions. But wouldn't this be too complex?

Initially, my feeling was that the default scheme should use the same system. Rather than being hard-coded as a separate function, it would be specified as the default configuration. The advantage of this is that it means less code to maintain and that it would be extremely simple to make minor tweaks once there is a GUI< since the GUI would show the default configuration already.

As you point out, this does present issues for localisation. However, there are other cases where locale specific default configuration might be useful, so perhaps this is another case for that.

Still, if we do go with a template based approach, perhaps you're right that it's better to extend rather than incorporate all of the logic. However, this isn't that relevant until we answer the questions above.

[nvaccessAuto]

Comment 9 by aliminator (in reply to comment 8) on 2014-09-11 11:43

I am not sure whether all users will agree to the same presentation scheme. If this is the case it should be definitely hard-coded and used as default.

I think the first step, then, is to work out what users actually want here. It's very possible some users would prefer slight deviations, but these are not the primary concern. The bigger question is whether there are major changes desired by most users. Even if there are significant differences of opinion, it may still be better to implement smaller options (e.g. a check box to display states first) rather than a template based system.

As an example #2955 is requested by several users. Yes, it can be incooperated easily. But it is not configurable (according to the patch). So, if some users do not agree to that scheme they would request a corresponding option. So it has to be made configurable etc. etc. Is it not better to have much work with a template system at once instead of having many tickets opened for small changes? Tickets such as #1955, #2346 and #4444 could then be closed and users only need to configure corresponding settings. Hence, from those tickets mentioned it should be clear which kind of modifications users are requesting.

The problem with a template based system such as proposed here is that it is extremely difficult to take all possible requirements into account. Furthermore, once it's implemented, it's very hard to support additional requirements later without breaking compatibility with the original scheme. For example, the current patch can't handle the following:

• Dots 7 and 8 for selected list items (role as well as state). This is a request that has been made by several users, so it is particularly relevant.
• A single abbreviation covering multiple conditions; e.g. vlnk for visited link, h1 for heading level 1.

These are just examples; there would be more cases it can't cover. My aim isn't to devalue your work at all. The point is that something which appears to be extremely flexible sometimes isn't that flexible or future proof in the longrun.

The second item mentioned can be done by assigning two same values to corresponding roles. But the first point is indeed difficult to solve and cannot performed by the patch. Yes, a templating engine would be the most flexible way and should cover most of the needs. If we do it on a per-role basis it should be extendable in the future even though the issue of showing dots 7 and 8 when an item is selected is released.

There might be another solution, not that flexible though but might integrate better with the existing infrastructure. We merge all settings/elements which can be toggled such as lists, links and tool tips in one list. Then, for each element it should be possible to set "on" or "off" for braille and speech separately (could be a solution for #1885). So, for example reporting of lists can be set to: Nothing (no reporting), speech, speech and braille and braille. As soon as more roles are needed they can be added easily. And, if more customisation is required a template engine might be added later. In that case I would leave the default configuration at its initial state except of applying the patch from #2955 to have the checkboxes on the left-hand side.

[nvaccessAuto]

Comment 10 by aliminator on 2015-06-21 08:25 What if we provide an interface for addons where implementation details are left to the addon developer and the presentation scheme in the core is untouched?

[ehollig]

I would be interested to see Braille users opinions on this issue. CC @derekriemer @leonardder and others

[ruifontes]

For me, since the major enhancements in Braille presentation in 2017.3, I will wait by users comments...

Rui

-----Mensagem Original----- De: Ethan Holliger Data: 15 de agosto de 2017 00:11 Para: nvaccess/nvda Cc: Subscribed Assunto: Re: [nvaccess/nvda] Introduce role-based braille mapper (#4105)

I would be interested to see Braille users opinions on this issue. CC @derekriemer @leonardder and others

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[LeonarddeR]

cc @dkager @bramd Personally, I'm for consistency, so #7232 would do the job for me and I'm not a big fan of having separate configurable property orders for every role. IMO, this would make the system way too complicated. However, I recall @bramd had a good use case for this.

[Adriani90]

@bramd could you please Elaborate on the use case @leonardder mentioned above? In the german community, which in my opinion is one of the biggest community using Braille Displays worldwide, People are requesting such Features for very Long time. Especially placing controltype and Status to a desired Location and deciding if they are displayed at all are the most wanted Features. Many users of Jaws and Cobra also want points 7 and 8 for selected items and for spell check. And lots of users, especially in professional Environments, want to customize the Location where a line break should be applied on a Braille Display (i.e. some People want items to be shown under each other on a Braille Display even though they are displayed from left to right on the Screen). This last example is a common practice for many users who are working in call centres or so. Many of them were Cobra users because there were coresponding scripts for Cobra which did this line break on the Braille Displays according to user's Needs.

[bdorer]

Could someone upload brlmapper.patch noted in https://github.com/nvaccess/nvda/issues/4105#issuecomment-155319764 here? @Andre9642 is interested. cc @michaelDCurran @Qchristensen

[taymun26]

I should have this on another machine. As soon as I find it# I will upload it here.

[bdorer]

Hi @taymun26,

did you have any chance to look for the trac attachment?

thanks

[lukaszgo1]

Old attachment from trac: brlmapper.patch.zip