notator / ResidentSynthHostTestSite

Host application for developing the GUI-less ResidentSynth
MIT License
2 stars 1 forks source link

Unable to connect my MIDI output to your MIDI input #1

Closed infojunkie closed 1 year ago

infojunkie commented 1 year ago

Hello,

I am building a Web-based score player, very similar to your work in this area. Here's my demo: https://blog.karimratib.me/demos/musicxml/

I wrote a pared-down web audio synth using surikov's webaudiofont module, but I wouldn't mind swapping it out for yours which looks more complete.

However, when I open your demo app at https://james-ingram-act-two.de/open-source/ResidentSynthHost/host.html, and select Midi Through Port-0 as input, and the same as output on my demo, I get no sound out.

If I output to the DX7 emulator instead https://mmontag.github.io/dx7-synth-js/, I do get sound.

On your demo app, I see a warning about a deprecated feature related to Web MIDI, screenshot below. I suspect Web MIDI is not initializing properly on your page.

Screenshot from 2023-08-04 23-45-31

Can you advise?

Thanks!

notator commented 1 year ago

Hi infojunkie!

Thanks for your interest and questions.

The demo app was designed to look for hardware input and output devices, such as a keyboard plugged into the computer or the computer's sound system. I suspect that its not possible to connect two web applications in this way, but if your "Midi Through Port-0" appears in the Input Device selector, then it should work. Does it? You could find out more about what's going on in the debugger.

I'd be interested to know how you connected to the DX7 simulator, since I'd also like to do that from my AssistantPerformer web app. Are there any docs you could point me to?

You are very welcome to use my ResidentSynth if you want to: In the (volatile) testSite branch of this repository, I'm currently finalizing the README for a major upgrade of both the ResidentSynthHost and ResidentSynth. When I'm finished, I'll be merging the testSite into the main branch, and updating the stable demo app. Instructions as to how to install the ResidentSynth on any web page are already in the testSite README.

The "Deprecated Feature Used" warning happens simply because the app is using the Web MIDI API (which is not, as far as I know, deprecated). This is not a Web MIDI initialization error. This behaviour (and the message) depends on the browser. (I mostly use Chrome, which also puts up a rather alarming message, but suppresses it once the user agrees to use the site.) The Web MIDI API is only used by the ResidentSynthHost, not by the ResidentSynth, so installing the synth on your site would not lead to such a warning.

Hope that helps! :-)

notator commented 1 year ago

Hi again!

I think I've found the way to connect my AssistantPerformer to the DX7 simulator. The AssistantPerformer has a MIDI output device port selector with which I can select my attached E-MU keyboard ( ! ). This means that I should be able to use the keyboard as a MIDI-Thru device, whose port can connect to the DX7's MIDI input port via its "MIDI device" selector. Unfortunately, the AssistantPerformer's "E-MU keyboard" output device is currently marked as "closed" in the debugger when I try to use it to send messages, so it doesn't work. That implies, of course, that I've got to open it first! That's something I'll try very soon...

Could something similar be preventing MIDI messages from getting from your application into the ResidentSynthHost ? Perhaps you could take a look in the debugger? I don't see any problems with doing so in principle.

Many thanks for the feedback. This is the way to make progress! :-)

infojunkie commented 1 year ago

Here's a video illustrating the expected communication between Web MIDI producer and consumer. My score player emits MIDI messages that it sends to a MidiOutput port as detected via the Web MIDI API. The DX7 emulator is listening on the same port and thus receives the MIDI messages. There's no necessity of involving hardware devices at any point in the workflow, and this is how I would like to use your ResidentSynth.

webmidi.zip

I will test your testSite branch and report back here.

infojunkie commented 1 year ago

Same result on testSite branch: no sound when I select Input Device: Midi Through Port-0 in your demo app. The Simple Input control does produce a sound.

I will debug if you don't have suggestions. Thanks!

notator commented 1 year ago

Thanks for the video. Yes, I've now really understood the problem.

I've been looking at my code for accessing MIDI input and output devices. It seems okay according to the Web MIDI API docs, but it was written before Promises were invented, so may be a bit flaky. For example, the open() and close() functions on MIDI devices now each return a promise that needs waiting for.

Strangely, my AssistantPerformer did once play the DX7 (i.e. using my keyboard as a MIDI-thru device), but I couldn't repeat the behaviour. Something (I don't know what) is not being set up properly. The MIDI-Thru device is at least now consistently open when its send() function is called. All my available output devices work fine when they are not MIDI-thru. Very odd.

This behaviour is the same in both Chrome and Firefox, so its probably not a browser issue. Which browser are you using?

Could you point me at the code you use for accessing/opening/closing your MIDI Through Port-0? Something might catch my eye...

All the best, and thanks again for the feedback. Very useful!

infojunkie commented 1 year ago

Commenting these 2 lines https://github.com/notator/ResidentSynthHostTestSite/blob/testSite/host.js#L325-L326 was enough to keep the port open and get some sound. It may be that the close() and open() methods need proper Promise handling.

However, what I heard is not what I was expecting to hear :-) so I will let you take a listen and tell me what you think.

notator commented 1 year ago

Done. Does that help? I can't test it from here, since I don't have a working/reliable MIDI-thru device.

If there's still a problem, I could try just keeping the device open all the time...

The host's default sound is an electric organ. You could choose a different one. If you copied the synth to your site, you could choose the available sounds (and defaults) yourself (from Surikov's presets).

infojunkie commented 1 year ago

Thanks, the updated testSite branch works out of the box :partying_face:

Regarding the sounds, I am expecting MIDI Program Change messages that are sent to the synth to be honored and result in loading the appropriate sounds. Is this not what MIDI synths are expected to do?

I cannot set the sounds ahead of time, since a Program Change message can occur at any time within the MIDI stream.

For example, my own webaudiofont-based synth traverses the MIDI file ahead of time to pre-load the instruments it will need. Upon a Note On message, I select the appropriate instrument and pass it to the player's queueWaveTable() method. https://github.com/infojunkie/musicxml-player/blob/main/src/WebAudioFontOutput.ts#L55-L107

notator commented 1 year ago

Great to hear that its now working. :-) But I'm still not happy with the code: I need to look at Promises again to see if I can improve my kludge. Also (just as important) I need to 1) revise the ResidentSynth's open() and close() functions so that they return promises, and 2) change host.js line 2106 (synth.open()) accordingly.

Regarding the sounds, I am expecting MIDI Program Change messages that are sent to the synth to be honored and result in loading the appropriate sounds. Is this not what MIDI synths are expected to do?

The ResidentSynth has configurable presets by design. One of the prime motivations was that I wanted to speed up loading times by only installing the presets required for an installation. If the web application only needs a piano, then it should not have to load a whole sound font. If you want/need a full, standard set of presets, then you could define one (in residentSynth/config/webAudioFontDefs.js.)

BTW: This project's terminology is currently rather confusing. I need/want to do a large refactoring job as follows: In fact, webAudioFontDefs.js really defines one webAudioFont with multiple banks, not multiple single-bank webAudioFonts. What I'm currently calling a WebAudioFont is, in fact, a Bank. My SOUND_FONT_INDEX message is really a BANK message. Standard BANK messages should work as expected!

notator commented 1 year ago

FYI:

I need to look at Promises again to see if I can improve my kludge. Also (just as important) I need to 1) revise the ResidentSynth's open() and close() functions so that they return promises, and 2) change host.js line 2106 (synth.open()) accordingly.

I've now done all that and it seems to be working, so the testSite branch and its web page are going to be stable for a while.

I now want to do the major refactoring, and will be doing it in a new "refactoring" branch on a different web page.

All the best, and thanks very much for raising this Issue. It helped a lot!

infojunkie commented 1 year ago

Thanks for your help. Closing this issue.