BLE connection interval setting

[johnty]

wonderful work on the BLE library and examples! thanks @nkolban for all the work!

i've been playing around with BLE MIDI with the ESP32 am having trouble getting Apple devices (iOS or OSX) to see the interface, working off this example. with lightblue explorer on iOS i can see the service and characteristics being added correctly, and it even showed up once in iOS as a BLE midi port (but then the device went "offline", and was never searchable again after that). i wonder if its because the default BLE connection intervals may be too high to be under the 15 ms spec as outlined by Apple.

some pointers on where i could insert the connection interval parameters (by perhaps modifying the existing BLE library files) would be much appreciated! i expect this might also be a setting that is worth exposing in the future perhaps?

thanks again,

johnty

[chegewara]

You can try to modify those lines to see if this is interval issue: https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/BLEAdvertising.cpp#L42-L43 https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/BLEAdvertising.cpp#L53-L54

[nkolban]

@johnty Oooh ... that may be a fun project. Can you elaborate on "BLE MIDI"? Are there any reference papers or specifications on what the interface looks like? What hardware/devices/technology would I need to recreate tests?

[johnty]

@chegewara: that looks promising. Will try it when I get a chance to next.

@nkolban the linked example above should work with any esp32 board without any additional hardware. It emulates a keyboard and presses the middle c key every 2 seconds :). To test it you can use any iOS app that supports midi (such as midi wrench), or OSX’s audio/midi setup utility. I’m on the road right now and hard to pull links but Apple has some docs on the format, and it also exists on midi.org

[nkolban]

Ahhh my friend ... you assume I am wealthy enough to own Apple products :-) ... I need to see what is available fro an Android phone or Windows 10/Linux PCs.

[johnty]

Haha, know what you mean. I actually don’t own much apple stuff myself personally and it’s mostly devices from work/research lab.

BLE MIDI should be available on win10/Linux too. In fact the above example had only been tested in those environments according to the author, which was why I initially guessed it was Apple’s more stringent latency requirements that may be the reason it’s not working on iOS/OSX...

[johnty]

quick update on this:

it looks like the issue here may not be the parameters in question (although @chegewara's tip may be useful later on esp. if tweaking of performance is needed), but instead a potential bug/undesired behaviour in OSX (and maybe iOS too, based on what i'm experiencing). basically, if i initiate the connection using another method then the ports seem to work fine.

incidentally, the setup process in Windows was quite smooth. too bad BLE MIDI ports aren't as transparent as in OSX which means applications that haven't been updated to BLE MIDI cannot see the ports without some additional routing via a third party app.

thanks for the help! (and once again, @nkolban's awesome work that made all this possible!)

[nkolban]

Studying this now ... https://www.midi.org/specifications/item/bluetooth-le-midi

It also seems that there is a device called the VS1053B which can decode Midi. Seems to be about $7.

[johnty]

MIDI is kinda like CAN bus in that its a general standard that deals with everything from how data is produced and consumed (by instruments/synthesizers etc) to the physical electrical specifications of the signals. To get (non-wireless) MIDI signals out physically all you need is a serial output and a few resistors; on the input side you would need a simple opto-isolator but otherwise it's just a serial input.

In the more recent BLE implementation it's basically the wireless link that replaced the physical wire, although an additional bunch of software has to exist on the host side as well to actually instantiate the MIDI port via bluetooth in the operating system so that it behaves as such.

[nkolban]

Is there any software on a PC (Windows or Linux) or Android phones which can received BLE/Midi data and "play it"?

[chegewara]

Did not check it but look promising: https://play.google.com/store/apps/details?id=jp.kshoji.blemidi.sample https://play.google.com/store/apps/details?id=com.mobileer.example.midibtlepairing

[johnty]

Windows 10: MIDIBerry supports BLE. I haven't dived into Android to see if the BLE MIDI implementation allows usage by "traditional" MIDI-enabled apps. For comparison in Windows you don't get automatic support for BLE MIDI while on OSX once you manage to pair with the device, any MIDI-capable app will be able to use it.

[danover]

Great work @nkolban ! @johnty , did you ever figure out what the actual issue with iOS/OSX was? On OSX I can get it to pair only in the Bluetooth Explorer app from XCode's Hardware IO tools... then it works fine, but for some reason the normal way of pairing BLE devices on OSX doesn't see the device (AudioMIDISetup). I still haven't managed to pair with iOS at all. Likely the same issue, something in BLEAdvertising::BLEAdvertising() probably isn't what Apple expects - any clues?

[johnty]

@danover, after playing with it further, it looks like the connection can be made consistently using Hardware IO tools in OSX. Very occasionally do I see that the service doesn't register after you hit "connect" in the BLE devices menu, and if that happens, you won't be able to add/enable the port in Audio Midi Setup.

I think the connection interval setting here is not critical either way, however. As when it works through the manual connection step, changing the value doesn't appear to make any difference. This of course assumes that the setting actually does something on the lower firmware level... :)

For iOS if I use LightBlue from PunchThrough and connect to the device there, the port seems to be accessible by other MIDI-enabled apps.

I'm not sure which side is exactly at fault here... For example Korg has a number of software for dealing with connecting to the BLE MIDI device. And on Windows it finds the device and connects fine.

[danover]

Thanks for the tip - on iOS, connecting first with LightBlue works as you said. Have you investigated Android further? I'm trying it with a Galaxy Tab3 now... no luck yet with either of the apps chegewara mentioned.

[johnty]

no luck with my Nexus 5... don't have any others to test with unfortunately.

totally off topic: are you going to be at NIME this year? (I'm a music tech student from McGill :)

[danover]

ah, OK. Hope to see you at NIME then!

[hoiberg]

Apparently, BLE MIDI devices are not supposed to show up in the iOS Settings app. Connecting has to be handled by the MIDI apps themselves, see this article. In Garageband, tap the wrench icon > Advanced Settings > Bluetooth Devices. There the ESP32 will show up reliably, and when connected it will be accessible to other MIDI apps as well. If the MIDI app you are using doesn't provide such an interface, you can also use a bridging app like midimittr or a BLE discovery app like LightBlue (as johnty mentioned).

An alternative way would be to enable pairing with this line:

pCharacteristic->setAccessPermissions(ESP_GATT_PERM_READ_ENCRYPTED | ESP_GATT_PERM_WRITE_ENCRYPTED);

Now you'll only have to use midimittr once, because the device will show up in the bluetooth settings under "Devices" (but it won't reconnect automatically, so this doesn't really help).

I was looking to replace the HM10 in my DIY MIDI keyboard with an ESP32, hoping that I'd no longer have to use an extra app to bridge the HM10 and CoreMIDI, but it turned out to be a dead end.

[chegewara]

If there is someone who can share esp32 midi code then i can try to investigate it (if i find any working windows app) when i have time. Its nice project and it would be nice to participate in it.

[johnty]

This is a good BLE MIDI example to start off with. it basically instantiates a BLE MIDI device and then transmits a note on/off pair at regular intervals.

This works in Windowns 10 (since Fall 2017 or so), but in Windows we have one additional complexity as I described above, which is that a BLE MIDI port is not transparent to end apps, and requires you to use the newer APIs. So if you want your BLE MIDI portto show up in a legacy MIDI app, you'll have to bridge the BLE port to a regular MIDI port. The app MIDIBerry will do it if you create a virtual (classic) MIDI port using something like loopMIDI, and from my poking around its based on the MIDI example application from Microsoft.

On the other hand, in Windows you can see BLE devices simply through searching for it so pairing is easier. How to re-connect to the BLE port after you pair it and turn it off seems a bit... mysterious as I've only been able to do so by removing and adding it again.

[chegewara]

Maybe someone will make use of this. http://protopage.com/hdop001 https://play.google.com/store/apps/details?id=hdop.btmo_demo https://play.google.com/store/apps/details?id=hdop.btmo

https://www.midi.org/specifications/item/bluetooth-le-midi https://www.silabs.com/community/wireless/bluetooth/knowledge-base.entry.html/2016/06/02/midi_over_ble-24Me http://wiki.lividinstruments.com/wiki/Bluetooth_LE_MIDI_Connection https://github.com/oxesoft/mourino https://www.midi.org/specifications/item/table-1-summary-of-midi-message

[chegewara]

Ok. First example code works with android. Client app: https://play.google.com/store/apps/details?id=net.volcanomobile.fluidsynthmidi

On windows 10 also is recognized with this app: https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/MIDI and there is no need loopMIDI. My test app is this code little bit modified: https://github.com/neilbags/arduino-esp32-BLE-MIDI/blob/master/BLE_MIDI.ino

Tested with midi player pro on windows 10 and works: https://www.microsoft.com/pl-pl/store/p/midi-player-pro/9nblggh5f0w8

also midiberry from windows shop: https://www.microsoft.com/store/apps/9n39720h2m05

PS There is something i have to mention. With midi player pro, when my windows is connected/paired with esp32 a note send from esp32 is played on windows, but when i minimize midi player pro esp32 is disconnected, and when i bring midi player pro back to front esp32 is automaticaly connected and sound is again played. I believe it is exactly what its expected.

Now i will try to decode midi file and send it from esp32 to windows. ( i think i will give up for now)

[johnty]

nice summary! thanks @chegewara for poking at it :)

i should stress about what "works" and doesn't in windows: the pairing/discovery seems to work fine, and once paired the BLE port is indeed present in the system. however, once you have a BLE MIDI port available, you need a program that uses the newer APIs to be able to see it. a lot of older programs that worked with "classic" MIDI ports (many existing sequencers and synth applications), will not be able to use it directly. so the workaround here is to use a program like loopMIDI to create a virtual "classic" MIDI port, and then a bridge app like MIDIBerry that can connect to both "classic" and BLE ports, and route data between them.

on OSX/iOS, and presumably Android, once you've paired and added the BLE MIDI device, you end up with a "regular" MIDI port that can be used by any MIDI app. so even an older MIDI program running on these systems, who may not know anything about BLE in particular, will be able to use the port!

[chegewara]

I am total lame in midi matter, i did few quick tests with applications i could easy find. I understand those arent pro musicians applications but i dont have any. From quick look at few informations about midi over ble ive found that secure bonded connection is required or at least one of existing midi device is using it and i believe it may help (just assumption).

If someone is interested i can show my base midi app with secure/bonded connection implemented. Like i said its modified this arduino code.

I like idea midi over ble with esp32 and i can cooperate to make it even better.

[johnty]

awesome! would love to see your example with secure/bonded connection. thanks again for your help and interest in this! :)

[chegewara]

https://github.com/nkolban/esp32-snippets/issues/510

[githufsb]

Glad to see this being actively looked at....just picked up a nodemcu board and trying to get this thing working. If I can assist with testing or anything please lmk!

[chegewara]

I have more knowledge about ble stuff and this library, so i can try to work with midi over ble. If there is something else i have to know about midi or you know any applications on windows 10 i can use to test just let me know.

[githufsb]

@chegewara you're amazing....issue #510 does it! I successfully connected to device via GarageBand in iOS, and it's playing the piano track I created!

[githufsb]

i was about to give up and buy a feather board with ble support, but I can confirm BLE MIDI works with ESP32 board

[chegewara]

@githufsb I didnt expected it will work so easy, but if it really works then im glad you will stay with esp32 and we all can cooperate to make some nice midi class.

[johnty]

@githufsb using the updated example, were you able to find and pair with iOS without an external app/extra step, or did you use LightBlue as per comments above?

[githufsb]

Yes, worked directly in Garage Band via settings/wrench->Advanced->Bluetooth Midi Devices. I didn't try yet with lightblue, but expect it would work too. It was the security bit added by chegewara that allowed pairing I think.

[githufsb]

If it wasn't obvious already, @chegewara made my day yesterday. And thanks much to @nkolban for the great work leading up to this....we should merge in sample from #510 to this repo and https://github.com/neilbags/arduino-esp32-BLE-MIDI (@neilbags I tried with that one, and it didn't work) when there's a chance.

[chegewara]

You can work with midi over ble now, one gangsta less on the street ;)