Playing live events with more than one button at the same time (multitimbral keyboard)

[Riobener]

Apologies for the noob question, your sample project explains how to play live event on a single button. What do I need to do if there are, for example, 3 such buttons, which all pressing at the same time, and it should play a chord, with the sound of the selected synthesizer? I tried many of my methods, re-read your wiki, and eventually ran into the problem of an excess of memory or the fact that at one time, the synthesizer only plays the last key pressed. P.S. sorry for bad english #

[igorski]

Hi there, good question! What you need to do is map your keybaord keys into down/up states (for instance on touch events) and map each unique key to a SynthEvent. I have added this page to the Wiki with some additional information.

[Riobener]

Thank you very much! Really appreciate your answer. I will definitely try to do it later.

[igorski]

What if you add debug logging to the onKeyDown and onKeyUp handlers ? Visually it looks like the key up and key down should equal the highlighted state of the keys (I'm not familiar with the library). But the audio sounds like the previous note is killed. Or maybe (depending on fidelity during recording), new notes are played but they are all at the same pitch (there is some audible phase drift). Can you log the event.getFrequency() just before event.play() ?

[Riobener]

Ah, sorry, i accidently delete my previous message. Yeah, i tried debug. Log is here:

2021-03-29 15:54:50.787 30304-30304/com.riobener.audiostudio D/T: KeyNumberDown = 4
2021-03-29 15:54:50.788 30304-30304/com.riobener.audiostudio D/T: event.getFrequency() = 164.814
2021-03-29 15:54:50.810 30304-30304/com.riobener.audiostudio D/T: KeyNumberDown = 7
2021-03-29 15:54:50.810 30304-30304/com.riobener.audiostudio D/T: event.getFrequency() = 195.9975
2021-03-29 15:54:50.812 30304-30304/com.riobener.audiostudio D/T: KeyNumberDown = 0
2021-03-29 15:54:50.812 30304-30304/com.riobener.audiostudio D/T: event.getFrequency() = 130.813
2021-03-29 15:54:50.883 30304-30304/com.riobener.audiostudio D/T: KeyNumberUp = 7
2021-03-29 15:54:50.883 30304-30304/com.riobener.audiostudio D/T: event.getFrequency() = 195.9975
2021-03-29 15:54:50.899 30304-30304/com.riobener.audiostudio D/T: KeyNumberUp = 4
2021-03-29 15:54:50.899 30304-30304/com.riobener.audiostudio D/T: event.getFrequency() = 164.814
2021-03-29 15:54:50.923 30304-30304/com.riobener.audiostudio I/ViewRootImpl@ee51a1d[MainActivity]: ViewPostIme pointer 1
2021-03-29 15:54:50.923 30304-30304/com.riobener.audiostudio D/T: KeyNumberUp = 0
2021-03-29 15:54:50.923 30304-30304/com.riobener.audiostudio D/T: event.getFrequency() = 130.813

I just pressed all three buttons at the same time(Chord C).

The original code is placing in the end of onCreate method of your example project. I didn't add anything else.

 // use PWM waveform (see enumeration)
        instrument.getOscillatorProperties( 0 ).setWaveform( 2 );
        ADSR synthEnvelopes = instrument.getAdsr();
        // set a positive release envelopes for a smoother fade out on noteOff
        synthEnvelopes.setReleaseTime( 0.25f );

        for (int i = 0; i < 13; ++i ) {
            int octave = (int) (BASE_OCTAVE + Math.ceil( i / 12 ));
            notes.add( new SynthEvent(( float ) Pitch.note(noteNames.get(i % 12), octave), instrument) );
        }

        pianoView = findViewById(R.id.pianoView);
        pianoView.addPianoTouchListener(new PianoTouchListener() {
         @Override
            public void onKeyDown(@NonNull PianoView piano, int key) {
                // custom to your view, get the key index corresponding to the event
                int index = key;
                // get the synth event
                SynthEvent event = getSynthEventForKeyIndex( index );
                Log.d("T", "KeyNumberDown = "+key);
                Log.d("T", "event.getFrequency() = "+event.getFrequency());
                if ( event != null ) {
                    event.play(); // let's heard some sweet sound
                }
            }
            @Override
            public void onKeyUp(@NonNull PianoView piano, int key) {
                // custom to your view, get the key index corresponding to the event
                int index = key;
                // get the synth event
                SynthEvent event = getSynthEventForKeyIndex( index );
                Log.d("T", "KeyNumberUp = "+key);
                Log.d("T", "event.getFrequency() = "+event.getFrequency());
                if ( event != null ) {
                    event.stop(); // trigger note end (initializes release phase)
                }
            }

            @Override
            public void onKeyClick(@NonNull PianoView piano, int key) {

            }
        });

What if you add debug logging to the onKeyDown and onKeyUp handlers ? Visually it looks like the key up and key down should equal the highlighted state of the keys (I'm not familiar with the library). But the audio sounds like the previous note is killed. Or maybe (depending on fidelity during recording), new notes are played but they are all at the same pitch (there is some audible phase drift). Can you log the event.getFrequency() just before event.play() ?

[Riobener]

What if you add debug logging to the onKeyDown and onKeyUp handlers ? Visually it looks like the key up and key down should equal the highlighted state of the keys (I'm not familiar with the library). But the audio sounds like the previous note is killed. Or maybe (depending on fidelity during recording), new notes are played but they are all at the same pitch (there is some audible phase drift). Can you log the event.getFrequency() just before event.play() ?

And it really looks like that the next pressed button is killing sound of previous. It works, if i creating, for example, new unique instrument for each button, but this is crazy solution which always crashes the application cause memory overflow. I'm not sure it is problem with piano keyboard that i found: link All features of your library works perfectly, but polyphony problem is what haunts me so far...

[igorski]

Not sure what is going on, the debug log you posted makes sense in the sequence of things though it's odd that the up events fire within 100 ms of the down events (sure it's possible but long presses make debugging easier).

I'm not around a development system for the next few days and I'll refrain from making comments on the PianoView library as I can't test it (and I don't want to imply that the "blame" lies there) but something is going awry between the interface and MWEngine. Indeed, creating an instrument per event is overkill, but should not lead to any memory issues as long as these are disposed appropriately (for the record: this should not be considered a solution).

I'd suggest trying to see if you can come up with a custom interface (just for the sake of testing) where you can map an individual pointer to an individual event and see how that works. I can see if I can create a POC inside the example activity as a viable example.

[igorski]

Actually a bug has crept in which affected SynthEvents using live playback. This has been addressed in d25df97ae7699db5f836be60677c18c0feeb54ff can you verify after pulling the latest changes whether multi timbral support now works for you ?

[Riobener]

Actually a bug has crept in which affected SynthEvents using live playback. This has been addressed in d25df97 can you verify after pulling the latest changes whether multi timbral support now works for you ?

Yeah, it finally works for me! I really appreciate your help and a fairly quick response. Thank you again :)