App does not receive any MIDI events

julienduchow commented 2 years ago

I just downloaded the sample app and installed it on an android device. Then I connected a piano with USB midi to my phone. The app does show the MIDI device and I am also able to connect to it.

But the app does not receive any input from my piano, e.g. pressing a key. I debugged it and saw that the connected variable of the device was true, but the connected variables of both input and output channel were both false, so I guess I was not really connected to the piano was I?

Do you have any idea what I am doing wrong? Thanks in advance.

mortenboye commented 2 years ago

Are you using the example app as is, or what does your setup look like?

julienduchow commented 2 years ago

Yes I used 1:1 your example app

mortenboye commented 2 years ago

The example app is very basic and only listens for CC commands. I dont know if you noticed.

It filters others types of midi out in the controller.dart class:

if (data.length >= 2) {
  var d1 = data[1];
  var d2 = data[2];
  var rawStatus = status & 0xF0; // without channel
  var channel = (status & 0x0F);
  if (rawStatus == 0xB0 && channel == _channel && d1 == _controller) { // <- 0xB0 == CC messages only
    setState(() {
      _value = d2;
    });
  }
}

I'm not sure how you debugged, but it should be possible to send and receive note on/offs as well as other midi commands

julienduchow commented 2 years ago

I know, but it does not even reach that part. I uncommented the following two comments:

print('init controller'); // <- This does get called
_rxSubscription = _midiCommand.onMidiDataReceived?.listen((packet) {
print('received packet $packet');  // <- This is never called

Also the variables of the channels were both false as I mentioned that should not be does it?

mortenboye commented 2 years ago

Are you testing on iOS or android? I'll try to test a similar setup here.

julienduchow commented 2 years ago

I tested on android, sadly I do not have a cable to test on iOS currently. Thank you very much for your help. :)

mortenboye commented 2 years ago

Just tested on both Android and iOS, keyboard into OTG Adapter/AppleCameraConnector respectively. On both devices the keyboard shows up. A tap on the listitem connects to it and a long press opens the controller page where I can receive CC commands and the UI responds, and I can see midi note commands in the log:

I/flutter (25616): received packet Instance of 'MidiPacket'
I/flutter (25616): data [145, 74, 47] @ time 420641124213334 from device ALESIS Vortex Wireless 2:45
I/flutter (25616): received packet Instance of 'MidiPacket'
I/flutter (25616): data [129, 74, 0] @ time 420641349427627 from device ALESIS Vortex Wireless 2:45

Not sure why it doesn't work your side. Can you provide more details?

julienduchow commented 2 years ago

What exactly do you mean with CC command? If I press a key on my keyboards thats also a cc command right?

Well hard to tell more details if I just used your sample app as is. I used my Android LG G6 smartphone and connected it via USB to my digital (an very old) piano. This midi works correctly as I already used it in other apps from the Playstore.

mortenboye commented 2 years ago

Usually a key only sends NoteOn and NoteOff messages. CC (Continuous Controller) are sent by knobs and faders, if your piano has any if those. But all midi should be received by the handler function in the sample app - It only just reacts on CCs. As you can see in the log on my previous comment, NoteOn and NoteOff messages are printed from within the handler. ([145, 74, 47] and [129, 74, 0])

julienduchow commented 2 years ago

I just tested on another Android device, but same result.

Then I tested a Native Java implementation: https://github.com/philburk/android-midisuite https://play.google.com/store/apps/details?id=com.mobileer.example.midiscope

There everything works fine.

julienduchow commented 2 years ago

I just reloaded your sample app from the dev branch and this time I really changed not a single line code. My process was the following:

Started the App in Debug mode on the device
Connected the USB cable from the Midi Keyboard into the phone
The app showed that new midi keyboard
Clicked on it short to connect
Clicked long on it to open the control page
Pressed some keys but not logs were genered

Do I use your sample app correctly? Below I attached the logs maybe you see a prolbem there? Im still wondering about this log line: D/FlutterMIDICommand(25338): list [{name=Mobileer MIDI Scope, id=3, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[]}, {name=USB MIDI Interface, id=6, type=native, connected=true, inputs=[{id=0, connected=false}], outputs=[{id=0, connected=false}]}] as it says that the device is connected but not the channels.

Logs:

Launching lib\main.dart on SM T835 in debug mode...
lib\main.dart:1
Warning: Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01
Warning: Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01
√  Built build\app\outputs\flutter-apk\app-debug.apk.
Connecting to VM Service at ws://127.0.0.1:60119/wfVE7KPYH8g=/ws
2
D/FlutterMIDICommand(25338): FlutterStreamHandler onListen
D/FlutterMIDICommand(25338): devices [Landroid.media.midi.MidiDeviceInfo;@7208a1e
D/FlutterMIDICommand(25338): list [{name=Mobileer MIDI Scope, id=3, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[]}]
W/Gralloc3(25338): mapper 3.x is not supported
I/flutter (25338): bluetooth state change BluetoothState.unknown
D/FlutterMIDICommand(25338): devices [Landroid.media.midi.MidiDeviceInfo;@42422ff
D/FlutterMIDICommand(25338): list [{name=Mobileer MIDI Scope, id=3, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[]}]
D/OpenGLRenderer(25338): makeCurrent EglSurface : 0x0 -> 0x7ddcd45300
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 0
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 1
D/ViewRootImpl@d569c21[MainActivity](25338): MSG_WINDOW_FOCUS_CHANGED 0 1
D/InputMethodManager(25338): prepareNavigationBarInfo() DecorView@ffbfaa5[MainActivity]
D/InputMethodManager(25338): getNavigationBarColor() -855310
D/InputTransport(25338): Input channel destroyed: 'ClientS', fd=86
D/SurfaceView(25338): onWindowVisibilityChanged(8) false io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452} of ViewRootImpl@d569c21[MainActivity]
D/SurfaceView(25338): surfaceDestroyed callback.size 1 #2 io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452}
D/SurfaceView(25338): remove() io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452} Surface(name=SurfaceView - com.invisiblewrench.fluttermidicommand_example/com.invisiblewrench.fluttermidicommand_example.MainActivity@52ffa07@0)/@0x646c8cc
D/OpenGLRenderer(25338): makeCurrent EglSurface : 0x7ddcd45300 -> 0x0
D/ViewRootImpl@d569c21[MainActivity](25338): Relayout returned: old=(0,0,1600,2560) new=(0,0,1600,2560) req=(1600,2560)8 dur=5 res=0x5 s={false 0} ch=true
D/ViewRootImpl@d569c21[MainActivity](25338): stopped(true) old=false
D/SurfaceView(25338): windowStopped(true) false io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452} of ViewRootImpl@d569c21[MainActivity]
D/SurfaceView(25338): onWindowVisibilityChanged(4) false io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452} of ViewRootImpl@d569c21[MainActivity]
D/ViewRootImpl@d569c21[MainActivity](25338): Relayout returned: old=(0,0,1600,2560) new=(0,0,1600,2560) req=(1600,2560)4 dur=12 res=0x1 s={false 0} ch=false
D/ViewRootImpl@d569c21[MainActivity](25338): stopped(false) old=true
D/SurfaceView(25338): windowStopped(false) false io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452} of ViewRootImpl@d569c21[MainActivity]
D/ViewRootImpl@d569c21[MainActivity](25338): stopped(false) old=false
D/SurfaceView(25338): onWindowVisibilityChanged(0) true io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452} of ViewRootImpl@d569c21[MainActivity]
D/ViewRootImpl@d569c21[MainActivity](25338): Relayout returned: old=(0,0,1600,2560) new=(0,0,1600,2560) req=(1600,2560)0 dur=7 res=0x7 s={true 542776016896} ch=true
D/OpenGLRenderer(25338): createReliableSurface : 0x7eee9ee0c0(0x7e5ff8c000)
D/SurfaceView(25338): surfaceCreated 1 #8 io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452}
D/SurfaceView(25338): surfaceChanged (1600,2452) 1 #8 io.flutter.embedding.android.FlutterSurfaceView{52ffa07 V.E...... ........ 0,0-1600,2452}
D/OpenGLRenderer(25338): makeCurrent EglSurface : 0x0 -> 0x7e5fd4aa00
D/ViewRootImpl@d569c21[MainActivity](25338): MSG_WINDOW_FOCUS_CHANGED 1 1
D/InputMethodManager(25338): prepareNavigationBarInfo() DecorView@ffbfaa5[MainActivity]
D/InputMethodManager(25338): getNavigationBarColor() -855310
D/InputMethodManager(25338): prepareNavigationBarInfo() DecorView@ffbfaa5[MainActivity]
D/InputMethodManager(25338): getNavigationBarColor() -855310
V/InputMethodManager(25338): Starting input: tba=com.invisiblewrench.fluttermidicommand_example ic=null mNaviBarColor -855310 mIsGetNaviBarColorSuccess true , NavVisible : true , NavTrans : false
D/InputMethodManager(25338): startInputInner - Id : 0
I/InputMethodManager(25338): startInputInner - mService.startInputOrWindowGainedFocus
D/FlutterMIDICommand(25338): device added MidiDeviceInfo[mType=1,mInputPortCount=1,mOutputPortCount=1,mProperties=Bundle[{manufacturer=null, alsa_device=0, product=USB MIDI Interface, name=USB MIDI Interface, serial_number=null, version=2.50, usb_device=UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=64514,mProductId=257,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=null,mProductName=USB MIDI Interface,mVersion=2.50,mSerialNumberReader=android.hardware.usb.IUsbSerialReader$Stub$Proxy@719d715,mConfigurations=[
D/FlutterMIDICommand(25338): UsbConfiguration[mId=1,mName=null,mAttributes=128,mMaxPower=50,mInterfaces=[
D/FlutterMIDICommand(25338): UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=1,mSubclass=1,mProtocol=0,mEndpoints=[]
D/FlutterMIDICommand(25338): UsbInterface[mId=1,mAlternateSetting=0,mName=null,mClass=1,mSubclass=3,mProtocol=0,mEndpoints=[
D/FlutterMIDICommand(25338): UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=8,mInterval=0]
D/FlutterMIDICommand(25338): UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=8,mInterval=0]]]], alsa_card=2}],mIsPrivate=false
I/flutter (25338): setup changed deviceFound
D/FlutterMIDICommand(25338): devices [Landroid.media.midi.MidiDeviceInfo;@4c72a2a
D/FlutterMIDICommand(25338): list [{name=Mobileer MIDI Scope, id=3, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[]}, {name=USB MIDI Interface, id=6, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[{id=0, connected=false}]}]
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 0
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 1
I/flutter (25338): connect
D/FlutterMIDICommand(25338): connect to native device: 6
D/FlutterMIDICommand(25338): open device MidiDeviceInfo[mType=1,mInputPortCount=1,mOutputPortCount=1,mProperties=Bundle[{manufacturer=null, alsa_device=0, product=USB MIDI Interface, name=USB MIDI Interface, serial_number=null, version=2.50, usb_device=UsbDevice[mName=/dev/bus/usb/001/002,mVendorId=64514,mProductId=257,mClass=0,mSubclass=0,mProtocol=0,mManufacturerName=null,mProductName=USB MIDI Interface,mVersion=2.50,mSerialNumberReader=android.hardware.usb.IUsbSerialReader$Stub$Proxy@e21f01b,mConfigurations=[
D/FlutterMIDICommand(25338): UsbConfiguration[mId=1,mName=null,mAttributes=128,mMaxPower=50,mInterfaces=[
D/FlutterMIDICommand(25338): UsbInterface[mId=0,mAlternateSetting=0,mName=null,mClass=1,mSubclass=1,mProtocol=0,mEndpoints=[]
D/FlutterMIDICommand(25338): UsbInterface[mId=1,mAlternateSetting=0,mName=null,mClass=1,mSubclass=3,mProtocol=0,mEndpoints=[
D/FlutterMIDICommand(25338): UsbEndpoint[mAddress=129,mAttributes=2,mMaxPacketSize=8,mInterval=0]
D/FlutterMIDICommand(25338): UsbEndpoint[mAddress=2,mAttributes=2,mMaxPacketSize=8,mInterval=0]]]], alsa_card=2}],mIsPrivate=false
D/FlutterMIDICommand(25338): onDeviceOpened
D/FlutterMIDICommand(25338): connectWithHandler
D/FlutterMIDICommand(25338): inputPorts 1 outputPorts 1
D/FlutterMIDICommand(25338): Open input port
D/FlutterMIDICommand(25338): Open output port
D/FlutterMIDICommand(25338): Opened device id 6
D/FlutterMIDICommand(25338): device status changed mInputPortOpen=[true] mOutputPortOpenCount=[0]
D/FlutterMIDICommand(25338): update device status mInputPortOpen=[true] mOutputPortOpenCount=[0]
D/FlutterMIDICommand(25338): device status changed mInputPortOpen=[true] mOutputPortOpenCount=[1]
D/FlutterMIDICommand(25338): update device status mInputPortOpen=[true] mOutputPortOpenCount=[1]
I/flutter (25338): setup changed mInputPortOpen=[true] mOutputPortOpenCount=[0]
I/flutter (25338): setup changed mInputPortOpen=[true] mOutputPortOpenCount=[1]
D/FlutterMIDICommand(25338): devices [Landroid.media.midi.MidiDeviceInfo;@14dcf91
D/FlutterMIDICommand(25338): list [{name=Mobileer MIDI Scope, id=3, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[]}, {name=USB MIDI Interface, id=6, type=native, connected=true, inputs=[{id=0, connected=false}], outputs=[{id=0, connected=false}]}]
I/flutter (25338): device connected async
I/flutter (25338): setup changed deviceConnected
D/FlutterMIDICommand(25338): devices [Landroid.media.midi.MidiDeviceInfo;@e85bef6
D/FlutterMIDICommand(25338): list [{name=Mobileer MIDI Scope, id=3, type=native, connected=false, inputs=[{id=0, connected=false}], outputs=[]}, {name=USB MIDI Interface, id=6, type=native, connected=true, inputs=[{id=0, connected=false}], outputs=[{id=0, connected=false}]}]
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 0
D/FlutterMIDICommand(25338): FlutterStreamHandler onListen
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 1
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 0
D/ViewRootImpl@d569c21[MainActivity](25338): ViewPostIme pointer 1

mortenboye commented 2 years ago

D/FlutterMIDICommand(25338): device status changed mInputPortOpen=[true] mOutputPortOpenCount=[1]
D/FlutterMIDICommand(25338): update device status mInputPortOpen=[true] mOutputPortOpenCount=[1]

This indicates to me that the ports do in fact open. But why you can't receive any data is not clear to me.

nzoschke commented 2 years ago

I'm having a similar challenge here. The example app shows my midi device but when I go into the controller screen there are no debug logs for midi packets.

Steps to reproduce...

Controller: Korg nanoKey Studio Device: Kindle Fire HD 10

Turn on Korg nanoKey Studio
Pair via bluetooth in device settings
Open Flutter Midi Command example app and scan for devices and don't see my device there. This is a whole other issue...

Then I found a big hack/workaround with another app that explicitly supports the Kor nanoKey Studio: G-Stomper Rhythm.

Open G-Stomper rhythm. This shows: "New MIDI device connected. Do you want to use nanoKey Studio' with G-Stomper?". Hit yes. I see the midi event light up on keypresses
Open Flutter Midi Command example app. Now I see "nanoKey Studio"!
Go into the controller page. No packet debug logs are displayed on key presses

I see this in the logs, so perhaps that's the underlying issue:

D/FlutterMIDICommand(12467): connect to BLE device: A0:CC:2B:52:26:81
D/FlutterMIDICommand(12467): Device not found A0:CC:2B:52:26:81

However if I run "Midi Scope" https://github.com/philburk/android-midisuite can connect to the nanoKey Studio and see midi notes no problem.

So I am working to debug...

Why doesn't nanoKey Studio device show up in MidiCommand().devices on its own?
What is G-Stomper doing to detect and initialize the midi device so that it does show up in MidiCommand().devices
Why "Device not found" error connecting to device?
Why no packets in onMidiDataReceived callback?
What is https://github.com/philburk/android-midisuite doing differently

I understand how difficult this is to debug for multiple devices. Just sharing my findings...

nzoschke commented 2 years ago

The device shows up in BluetoothAdapter.getBondedDevices() since I paired it in the device Settings.

https://developer.android.com/reference/android/bluetooth/BluetoothAdapter#getBondedDevices()

I can pass that into MidiManager.openBluetoothDevice to initialize it

https://developer.android.com/reference/android/media/midi/MidiManager#openBluetoothDevice

nzoschke commented 2 years ago

A patch to add support for my device is https://github.com/InvisibleWrench/FlutterMidiCommand/pull/61

geeetarguy commented 1 year ago

Okay, so I don't know if anyone has made this work since this thread fizzled out...but, let me explain the situation I'm seeing: I am getting things working fine in iOS and macOS apps outbound. I'm specifically testing right now with Logic Pro and am able to get it to receive midi messages from my Flutter app. Awesome. Very cool.

But...I'm trying to receive midi time code from Logic and am not having it work for me. What I'm seeing is that if I use a 3rd party midi monitor, Logic is sending time code. Awesome. It looks like Flutter is receiving time code. Awesome. But, the example app code does not trigger at all. I'm putting breakpoints all around it and getting nothing triggered other than the init.

Here's what i'm finding: It looks like it is triggering in method_channel_midi_command.dart right around line 120. Like:

/// Stream firing events whenever a midi package is received. /// /// The event contains the raw bytes contained in the MIDI package. @override Stream<MidiPacket>? get onMidiDataReceived { // print("get on midi data"); _rxStream ??= _rxChannel.receiveBroadcastStream().map<MidiPacket>((d) { var dd = d["device"]; // print("device data $dd"); var device = MidiDevice(dd['id'], dd["name"], dd["type"], dd["connected"] == "true"); return MidiPacket(Uint8List.fromList(List<int>.from(d["data"])), d["timestamp"] as int, device); }); return _rxStream; }

It looks like it's getting an exception hit here and the midi messages never make it into the example app.

I'm extremely new to Flutter. I've only been working with it for a few weeks. That said, it looks like this is possibly a little deeper into the codebase than I was expecting to go. It looks like this is part of the Flutter SDK? Or is it actually part of FlutterMidiCommand? I looked through

I wasn't able to find it in the FlutterMidiCommand codebase, though I did see a reference to it, which makes me thing it is lower level Flutter code.

Any guidance here would be fantastic. I'm more than happy to help out and get my hands a little dirty, but as I'm a little bit of a newb.

Any guidance would be fantastic. I'm guessing that I'm not the only person stumbling upon this? There has to be some other Apple devs wanting to grab time code from DAWs, I'm sure.

Thanks in advance for any help. Geeetarguy.

geeetarguy commented 1 year ago

So, I was able to grab Midi clock in from Logic. What I found is that method_channel_midi_command.dart at var device = MidiDevice( dd['id'], dd["name"], dd["type"], dd["connected"] == "true");

is not grabbing the name and throwing a null pointer exception. As a hack, I was able to just put a " " space in and it works, though it does not show which device is getting the clock from the logs. Also, it has to have the app's device selected, not Logic Pro Out as I would have expected.

I know this work around is in another project, but I thought I'd document what I found as a work around here.

If there is a better way of doing this where I can actually get the real name and not a space, that would be awesome.

mortenboye commented 1 year ago

@geeetarguy Thanks for the details and debugging. First I think, what you are describing is a different issue than the one this thread describes, so might be better to create a new issue to track this.

I just did a quick test with device/Mac connected in a Network Session, which works fine so I'm guessing you are connecting via bluetooth? This indicates that this is a bluetooth issue rather than a MIDI issue.

Also, it has to have the app's device selected, not Logic Pro Out as I would have expected.

Not sure what you mean by this. One device cannot "see" the apps on another device.

InvisibleWrench / FlutterMidiCommand

App does not receive any MIDI events #42