Enhancing Accessibility for Yamaha Expansion Manager

[gabrielschuck]

Overlay accessibility to Yamaha Expansion Manager

Hello! I would really appreciate it if the Yamaha Expansion Manager could be accessible through the reahotkey. It's a free desktop application for PC that allows you to create, install, and manage expansion packs, as well as transfer them to compatible Yamaha keyboards. I use it a lot in my daily life, although I have accessibility issues that greatly hinder me. One of my difficulties is creating my own packs: the interface is not accessible for inserting or deleting samples directly in it. As an alternative, I create my samples in the SFZ format and then convert them to SF2, which is the supported format. Even so, I occasionally need to delete or replace, for example, a looped sample with a non-looped one, and that's when I rely on the interface to do this. Another problem is when importing packs that need a fixed LSB bank to function correctly on the keyboard. This operation is also done through the interface, but not even a person with little technical knowledge who can see has an easy time helping, as it requires drag and drop, and this part of the interface is completely inaccessible. I even made a script in AutoIt to give me a bit more comfort and speed, but I must admit that I don't have advanced programming knowledge, and even so, it could be more useful if I could solve these problems. https://github.com/gabrielschuck/GS-YEM-Access. I hope you can think about this with great care and I thank you in advance!

[Timtam]

Hello there,

I think this will be pretty hard to do, as I think neither @MatejGolian nor me do have the software or a keyboard that goes along with it. We can certainly check out your AutoHotkey script and include the features into ReaHotkey, but apart from that, developing something without the possibility to test is basically not possible. We'll however keep it around, who knows what will happen.

[MatejGolian]

I took a look at the program and while it's possible to navigate the interface and activate certain elements with object nav, it's pretty hard to tell what's going on, since basically all buttons in the UI are unlabelled. So in short, I understand why you've decided to create an accessibility script, Gabriel. @Timtam, you're correct, I don't have a compatible keyboard either. That being said, owning one might not be absolutely required. The main problem here might actually be that we're not users of the program and thus not familiar with it. Gabriel, your script seems pretty comprehensive and porting it to ReaHotkey may not be straight forward/easy. In any case I have my doubts as to whether we could make more things accessible than you've already done. For example, I have no idea if it's at all possible to do something about that drag-and-drop issue. We could try to port your script to ReaHotkey, but we would need to know where to start/what to start with.

[gabrielschuck]

@MatejGolian and @Timtam: First, i apologize for the long post.

I can guide you within my capabilities and what I've discovered about the interface so far to adapt it to Reahotkey. I believe that based on the program's manual and with feedback from other users who own arranger keyboards (even if few), we could try something. A long time ago, I saw this post on a blog that was one of the reasons why I made this script, mainly for personal use: https://tanukispaceen.wordpress.com/2020/09/02/yamaha-expansion-manager-and-its-accessibility-for-the-blind-people/

As for where to start: For example, on the main screen with NVDA's object navigation, you should have seen two lists. One refers to the contents of a pack (instruments), and the other is the packs already imported. Here my script has two problems:

1. It doesn't always focus the keyboard on the correct list.
2. The contents of these lists are not read by the screen reader. When focused, NVDA only says "List item." You can navigate both lists with the arrow keys, but to find out the content, I have to use the Windows OCR item by item. I wasn't able to implement this in my script because I don't have the necessary knowledge for it. Furthermore, the way it clicks on unlabeled buttons and focuses on lists is very rudimentary. For example, it doesn't work with UI objects, which would be ideal for this kind of thing, and I don't even know how it would work with AutoIt.

I believe that implementing OCR to read the list items when porting to Reahotkey would be a good starting point. To test it, you would need the N27 from a compatible keyboard (I can provide mine if needed) and download a compatible pack from the Yamaha website. You wouldn't need to have the keyboard on hand. This N27 is generated by the keyboard itself and contains information about the model, expansion memory size for samples, etc.

In my script, the instructions to import an N27 and load a pack are as follows:

1. After the script and YEM are open, a message will be spoken asking to import the N27 file.
2. Press the F3 key. A submenu will open, navigate with the arrow keys to "Import Instrument Info."
3. The Windows Explorer window will open to locate the file. To verify if the file was imported, I added a shortcut, which is Ctrl+I. It retrieves this information from an ini file dynamically generated by the software as soon as the N27 is imported. It should appear somewhere in the interface, but that was the way I thought of implementing it.

Now we need to download a compatible pack. There are many on the Yamaha website. Let's use the "Alpen pack" as an example: https://download.yamaha.com/files/tcm:39-1366332/ Download the file and unzip it. One of the files has the cpf extension (content protected file). Back to YEM, with my script, press F1, and in the menu that opens, choose "import pack." Load the cpf file. Now the list of packs and the list of instruments in a pack will be filled. F7 goes to the pack's content list, and F6 goes to the imported packs list. All of this can be done with object navigation, but there are many unlabeled buttons, so you need to click on each one to find out what they do. That's how I made the script. I hope this has helped and given you some direction!

[gabrielschuck]

Speaking now about the possibility of creating our own packs, which is also possible:

Using the F1 key to open the menu, we have the option "Create pack." When the pack is created, in the pack list (F6), the pack we just created will appear with the name "pack001." We can change the name with the F2 key to open another menu, selecting the "rename" option. We type the desired name and press enter.

Now, creating a voice for this pack is the big problem. Usually, if the sighted user wants to create a voice from scratch, they will go to the "add content" menu (shortcut F4 in my script) and select the "create normal voice" or "create drum voice" option. Regardless of the option selected, an item will be created in the pack's content list (F7) with the default name "Normal Voice 001." We can change this name by accessing the "edit content" menu (F5). We move with the arrow to "rename" and press enter. We put the name we want. Now, selecting the item in the list, we can press F5 to access the "edit content" menu and choose the "edit voice" option. Here, the Voice editor window will open. As described to me, it shows a virtual keyboard at the top and, below, the sample associated with each key. It's possible to control the samples using any keyboard as a controller, including opening unprotected packs (ppf) to try out the voices before importing them into the keyboard, which is not possible for protected packs (cpf). For this, the shortcut F9 allows me to configure the audio output and MIDI input. However, unfortunately, it's not possible to add the waves, at least I couldn't do it with accessibility. That is, adding and associating each one with a keyboard key. It's not possible to do that directly. In voice editor I could only edit parameters like volume, velocity limit, etc. To display the parameters, you need to locate, with object navigation, the "common" checkbox.

So, what is the alternative in this case? In the "add content" menu (F4), there is the "import content" option. Pressing enter on this option will open the Windows Explorer window for you to locate a file supported by the software, and one of them is the SF2 format. I create my samples in SFZ format using Sforzando to play them, and then convert them to SF2 to make adjustments in the voice editor, option "edit voice" in the "edit pack" menu (F2). Although this helps me a lot, there is a problem. And this was a situation that recently happened to me:

I made an accordion sample in SFZ format and converted it to SF2. However, there were several non-looped wave files. I needed to replace these samples with other looped ones. In YEM, this would be done precisely by the voice editor, where there is no accessibility for this. The only alternative for now is to replace the looped waves that are included with the SFZ, convert to SF2, and import this SF2 again into the Yamaha Expansion Manager. Unnecessary work, isn't it?

That's right. This means that, otherwise, it is not possible to add the waves during the voice creation with accessibility, unfortunately. The user would have to have knowledge of SFZ and then convert them to SF2.

[gabrielschuck]

If, by any chance, you can't get my script to click on the unlabeled buttons that open these menus, or if it's going out of the YEM window: In YEM, press Alt+Space and then choose the "Minimize" option. Don't ask me how, but it has been solving it this way.

Those who can see say the software interface is very odd. Using NVDA's Insert+F1, I found out it seems to have been made using QT. Best regards, and I hope I've helped!

[MatejGolian]

Thanks, that helped indeed. I was hoping that AHK would be able to retrieve the list of controls, so that we wouldn't have to use mouse coordinates, but unlike NVDA, AHK does not find any controls in the window. So it seems that mouse coordinates are our only option.

[MatejGolian]

Gabriel, if you want, post an N27 file somewhere, so that we can determine whether the OCR thing could work. I have looked online, but probably there are none available to download.

[gabrielschuck]

I'll send the n27 from my keyboard here, it's a relatively small file. Yes, AutoIt also doesn't recognize the controls of the application. NVDA should detect them using object UI. I even found a code on the forum that enables AutoIt to work with object UI, but it's very confusing, I didn't understand how to use it: https://www.autoitscript.com/forum/topic/197080-using-ui-automation-code-in-autoit/

[gabrielschuck]

TYROS5_InstrumentInfo.zip

[MatejGolian]

Thanks for the file. I have looked into it and unfortunately, there are a couple of show-stoppers at the moment.

1. To be able to report which patch from a given pack is currently selected, we would need to know what area of the screen to OCR. The patch selector is (or at least seems to be) a list view displaying multiple items, but when you select an item, the order of those items doesn't seem to change - probably just the appearance of the selected item changes in some way. And since every item has its own screen coordinates, we would have to OCR the corresponding region of the screen - something that we're not able to determine at the moment.
2. Leading on from point number 1, interacting with the real controls of the window seems to be the only viable path forward. This library could possibly help us out here, although I haven't tested it yet.

[gabrielschuck]

@MatejGolian, I'm not sure if you managed to navigate through the pack content list with the arrow keys. From what I gather, when you click with the mouse on one of the lists, you can then navigate with the arrow keys on the keyboard. The selected item is the one with the arrow on it. If you press the delete key (for unprotected packs), it asks if you want to remove the item from the list and speaks its name. With object navigation, you can read the message. If you select another item with the arrows and press delete again, you'll know it's a different item. The same applies to imported packs. If you manage to navigate through the pack list with the arrows and press delete on any item, you'll know the name of the pack you're deleting. Therefore, I believe we would have to use OCR on the list itself, on the item that has the arrow on it. Here's a test: Try navigating through one of these lists with the arrow keys and, on any of the items, activate Windows OCR (insert+r). If your resolution is high enough, you should be able to read the focused item.

[MatejGolian]

You are probably correct, it's just that in order to be able to do anything ReaHotkey-wise, we must first be able to retrieve the lists and their items. In the meanwhile I had a look at that script I discovered. While I haven't tested it in practice, it looks promising, but integrating it into the existing ReaHotkey system will probably necessitate writing some new classes and making some changes to the existing code. And of course, prior to that we must first understand how that script works.

[MatejGolian]

I have tested the things you suggested, but over here the arrow keys do not move through the patches in the pack. I don't know why, so that's another problem. I have the 2.10.0 version of the Expansion Manager installed. That being said, moving through the items with objectnav and than using OCR gives fairly good results.

[gabrielschuck]

Yes, it's possible to navigate the list using object navigation. But here, if I hover the mouse over any item (insert+shift+m) and single-click, I can navigate the list with the arrow keys. Is that what you tried to do there?

[MatejGolian]

Yes, I did exactly that. Also this is with no accessibility scripts running.

[gabrielschuck]

What keyboard layout are you using? I use the laptop layout. In this case, I use Insert+[ to simulate the left click after moving the mouse with Insert+Shift+M (Insert+Enter doesn't work).

[gabrielschuck]

After finding the application objects with Insert+Shift+Right Arrow and then Down Arrow, I move to the right and navigate through the unlabeled buttons until I reach the first list. Once there, I press Insert+Shift+Down Arrow, Insert+Shift+M, and Insert+[

[MatejGolian]

The problem is that after I click one of the list items, the focus does not stay there. Instead the focus lands on one of the preceding buttons in the UI. When I press NVDA+Shift+Left, I get Alpen_T5 reported. Maybe the arrow keys do work if you stay focused in the list, but that does not work thus far over here.

[MatejGolian]

I created a little test in my Expansion Manager branch. If you open Expansion Manager with that version of the script running and than wait a little, you should get a fake Add instrument button announcement. When you press enter and again wait a bit, you should be able to navigate the menu with the arrow keys and for example import an instrument. I think that the basic concept kind of works, but for whatever reason, the script is terribly slow. Another problem is that I haven't found a way to focus the button without clicking it, which results in another set of problems. Honestly at this point I don't know if it's possible to make the script work acceptably with that program at all.

[MatejGolian]

I'll close this, because it seems that there's not much that can be done about it.