Import of 'javax.sound.midi.*' fails

[dfkettle]

I don't think this is a bug in Pd4P3, but I thought I should log it anyway, just to let you know. I tried to port your "MIDI Example" to Android mode, but I'm getting an error on this line:

1. ERROR in C:\Users\user\AppData\Local\Temp\android3179739350972919939sketch\app\src\main\java\processing\test\midi_example\MIDI_Example.java (at line 11)
    import javax.sound.midi.*;
           ^^^^^^^^^^^
The import javax.sound cannot be resolved

I guess the library isn't available in Android, and I'm not sure if anything can be done about it (maybe using some other MIDI library for Processing?). I'm running this on a Samsung Galaxy Tab Lite 6 under Android 12 and Processing 4.0.1. Attached is my version of the MIDI example, along with the log files.

221211_141423.err.txt 221211_141423.out.txt MIDI.pde.txt MIDI_Example.pde.txt Notes.pde.txt

[robertesler]

Thanks @dfkettle.
Yes, you are right, thank you! The Java Sound MIDI lib is not supported by Android. I'll have to look more into it but Android supports MIDI via the media API: https://developer.android.com/reference/android/media/midi/package-summary As you saw in the Pd4P3 example, I'm just using Java Sound's MIDI API, nothing related to the library itself. BUT that doesn't mean there shouldn't be an example in the Pd4P3 Android folder. I'll see I can whip this up some time soon. If you have any luck please feel free to share. Now I just have to find a MIDI controller that works with Android. : ) Thanks again for testing all this.

[dfkettle]

Using a USB OTG cable (and maybe an adaptor on one end), I've been able to connect both input (an Akai MPK mini keyboard) and output (a Korg NTS-1 synth) to my Android tablet using other apps. I think any MIDI device should work, with the appropriate cable. That's actually what I have in mind as a project, to develop an app with Pd4P3 so I can load presets to the NTS-1 and play it with the Akai keyboard, using my tablet to connect them (I'll need to use a USB hub to connect them both at the same time).

[robertesler]

That sounds pretty cool. I'm planning to look more into this next week. I'm fairly confident the Android media MIDI package will work, but haven't used it before.

[robertesler]

I've created a pretty reliable Android MIDI example now. It took a while because Android MIDI was not as straight forward as it looks. It works with a USB MIDI keyboard, if the keyboard isn't plugged in when app starts it may crash. I am working on a more versatile implementation for the actual release. You can find the example here: https://github.com/robertesler/Pd4P3/tree/main/examples/Android/AndroidMIDI I may create a separate Android MIDI library for Processing as there doesn't seem to be an easy way to use it directly unless you are using Android Studio. I'm closing this issue, if you do test it let me know if anything doesn't work.

[dfkettle]

It's probably something I'm doing wrong, or not doing at all, but I'm getting an error for a missing function, "startMIDI()", as well as some warnings about unused variables. I'm running Processing v4.2 under Windows 10 (I upgraded from Windows 7), with Pd4P3, v0.0.6, and Android Mode for Processing 4, v4.5.

[robertesler]

Sorry about that, there was a bad merge. It should just be midi.start(this.getContext(), midi); I've updated it. I'm still working on this for the distribution release so the unused variables will either get used or go away in the future. Thanks for trying it.

[dfkettle]

I'm able to run the sketch on my tablet now, but it doesn't seem to do anything except display the text"Note: 60 | 100" against a white background. I have a MIDI keyboard plugged into the USB port of my tablet, but nothing happens when I play it. I've used this setup with other Android apps, so I don't think there's any problem with the hardware.

I don't fully understand what the sketch is doing, but it should play a note using the Synth class when it receives a MIDI Note In, shouldn't it?

[robertesler]

Yes, right now the display probably isn't helpful. It should play a sawtooth synth with MIDI input. I would check if the audio is working on the sketch and test you are actually receiving MIDI input too into the MIDI class. It works on both my Android devices. Again this is super alpha right now. So lots more work to do.

[dfkettle]

I added some code to your library to display incoming MIDI messages on the screen, but it doesn't seem to be receiving any MIDI data (assuming my code to display the messages is correct). I didn't use "println()" because I don't know where to find the log file that it would write to on an Android device.

Which version(s) of Android did you test this on? I'm on Android v13. Sometimes, changes to Android can cause existing apps to break.

Here are the files I modified, if you want to merge them with your copy. You can find the lines I added by scanning for "// dfk 2023-04-18", there aren't very many. Or I can try creating a fork with my changes, although I've never done that before (I don't have much experience with GitHub).

AndroidMIDI.pde.txt MIDI.pde.txt

[robertesler]

Thanks for working on this. I actually did something similar yesterday too and committed it to a slightly more up to date version. I don't have Android 13 yet, so everything has been tested in Android 12 or lower. I don't doubt Android 13 could break MIDI, they are deprecating several things in API 33 (13+) but have yet to update their own code examples.
Mainly Android is updating for MIDI 2.0 which uses an entirely different packet system. I could try an emulator this week and see if I can figure out what is going on.
If you want to use println() to debug P4 sketches you can use wireless debugging, it's pretty much the only way to debug MIDI on Android. Here is how I did it: Plug your device into your machine. Make sure you know where the terminal or command prompt application adb is located. You can type adb and if it doesn't work then type the full path or add the path to your shell or environment variables. Then: adb tcpip 5555 adb connect your.ip.address of your device Finally check it worked: adb devices If it shows your device with :5555 at the end it worked. Unplug and code away. The P4 console should work while running the Android sketch.
The code I'm using is mostly via Google or Phil Burk and it seems to only be tested on Android 12 or lower. There has been some discussion of issues with 13+ but nothing yet that totally breaks MIDI. If you discover anything please let me know and I'll start a new issue. For now this example will stay in Alpha until it can be thoroughly tested.

[dfkettle]

Not having any luck with the console messages. I get the following error when I try to connect:

PS C:\Users\user\Documents\Processing\android\sdk\platform-tools> .\adb connect 192.168.0.39 cannot connect to 192.168.0.39:5555: No connection could be made because the target machine actively refused it. (10061)

In "Developer Options", I set "Wireless debugging" and "USB debugging" on. Maybe there's some other setting I have to turn on.

[robertesler]

I'm not exactly sure what that error means, have you tried this thread? https://appuals.com/no-connection-could-be-made-adb-error-10061/

Also, I tested this example on an Android 13 machine, Samsung Tablet, and it works as expected. I've added a device printout on the target display so you can see all the MIDI services and devices. If it is still not working you can check if the device you want to use is actually being used. Find the device id # and then manually enter it on line #178: MidiOutputPort outputPort = device.openOutputPort(finalMyDevice); Instead of finalMyDevice use the number from the list. I don't have any show up on my list other than my MIDI keyboard, so I can't tell if that could be a problem. I know other apps can be listed as a MIDI service so you may be opening a device/service other than your MIDI keyboard or intended device.

[dfkettle]

I haven't solved the console issue yet, but your latest change (to display the MIDI devices) helped me figure out what was causing the problem. I had an app called "FluidSynth" (a virtual MIDI synthesizer) installed on my tablet. It showed up as the first MIDI device in the list, and my MIDI keyboard showed up as the second device. By default, I guess your example sketch expects incoming MIDI messages to come from the first device. So I uninstalled FluidSynth and now I'm getting sound from your synth.

One problem is that the notes are getting "stuck". I think my keyboard sends "Note Off" messages rather than "Note On" messages with a velocity of zero. Looking at your code, the "NoteOff()" method is empty. I'll add some code to the method to call "Poly.perform()" with a velocity of zero. (It seems there's no method in the Poly class to handle "Note Off" events.)

[robertesler]

Just add noteOn(channel, noteIndex, velocity); to the noteOff(); method. Not all devices call a note off message, which the one I am using does not. I just added it to the example on github https://github.com/robertesler/Pd4P3/blob/main/examples/Android/AndroidMIDI/MIDI.pde.
You shouldn't need to touch Poly. It behaves essentially the same as [poly] in Pd, just without voice stealing.