Module Request: TouchOSC Bridge (MIDI + keystrokes)

[RickDDD]

I would be happy to see support for TouchOSC Bridge. TouchOSC Bridge supports MIDI and keystrokes via network which is both not new, but it 1) belongs to the Android/iOS app TouchOSC with ten thousands of users 2) brings its own virtual MIDI port, no extra installation necessary 3) is free 4) is configuration free 5) handles Midi AND keystrokes 6) provides Midi feedback over the same connection 7) does all of this with easy OSC messages so it should be easy to implement

I already found out the ports and reverse-engineered the used OSC messages and would be happy to help.

MIDI handling uses the OSC type tag 'm' which is not yet implemented in the generic OSC module (would be good to have this), but it is not very different from sending integers. I tried to send the hex data with generic tcp-udp and it works. SysEx is sent as string, this already works with generic OSC.

Keystroke handling sends a string and an integer via OSC, that also works with generic OSC module.

Please get in touch if you would like me to assist.

https://hexler.net/products/touchosc

[RickDDD]

@josephdadams

There is no official documentation, but as I wrote I reverse-engineered everything that is needed and got it 100% working with Generic OSC module and a bit of Generic TCP-UDP module (only needed because Generic OSC module lacks the 'm' type tag). So documentation is actually available.

I'm not new to programming in general, but I would have to start from nearly zero when it comes to companion modules. Don't want to sound lazy, but this would take me ages to learn and do it myself, but I'm pretty sure this would be a quick hack for someone with experience as the module only needs IP configuration and 3 (three) actions.

[josephdadams]

I marked it missing documentation because even though you reverse engineered it (great job!) I didn't see that included.

[RickDDD]

@josephdadams

Ok, so here is my documentation für MIDI, SysEx and keystrokes using TouchOSC Bridge. Keystrokes are sent to the active window (as usual), MIDI and SysEx are sent to the virtual MIDI port that TouchOSC Bridge adds during installation (also called 'TouchOSC Bridge').

TouchOSC Bridge app is configuration free, sits in taskbar and listens to UDP port 12101.

It accepts 3 different types of OSC messages, representing MIDI, SysEx and keystroke. If there are questions how to construct an OSC message, I can help.

1) SysEx

Least important, but easiest message. OSC message only containing a string, already possible with generic OSC module.

/sysex "(SysEx data)" -> string of hex data, always starting with F0, ending with F7 (everybody who wants SysEx knows how to use it)

2) Keystroke (no strings, one key at a time plus modifiers only)

OSC message containing a string and an integer, already possible with generic OSC module.

/key "a" 0 -> key down /key "a" 1 -> key up (has to follow or key (e.g. Shift) will stay pressed)

Modifiers:

/key "CONTROL+ALT+SHIFT+COMMAND+a" 0 -> order of modifiers seems to be unimportant, upper or lower case letters, too

special keys can be used as follows:

SPACE ENTER UP DOWN LEFT RIGHT ESC TAB BACKSPACE DELETE F1 - F20

3) MIDI

OSC message with one argument, type tag 'm', containing 4 bytes Hex data. Type tag 'm' not implemented in generic OSC module yet, but already possible by building and sending complete OSC message as UDP Hex data with generic TCP-UDP module.

/midi 00 7F 1C 90

The 4 bytes are: 00 - port id (always 00 for TouchOSC Bridge) 7F - data2 (here note velocity) 1C - data1 (here note number) 9&0 - status byte (message type (here note on) & channel (here 0))

I won't explain MIDI standard here, it is well documented, but I can help if needed. Please note though that the byte order in the OSC message used by TouchOSC Bridge deviates from the byte order in OSC specification. Probably by mistake, but doesn't matter.

For reference here the above MIDI OSC message as Hex data: 2f6d696469000000 2c6d0000 007f1c90 /midi ,m (MIDI bytes as explained above)

In generic UDP module: %2f%6d%69%64%69%00%00%00%2c%6d%00%00%00%7f%1c%90

TouchOSC Bridge understands the following MIDI message types: CC, Note On, Note Off, Poly Pressure, Program Change, Channel Pressure, Pitchbend, Start, Stop, Continue

---

Needed steps:

Add "Send MIDI (type tag 'm')" to generic OSC module. This would allow complete and quite easy operation of TouchOSC Bridge.

Or skip first step and: Take "Send string" and "Send message with multiple arguments" from generic OSC and put them in a new module. Add "Send MIDI (type tag 'm')". This would allow complete and quite easy operation of TouchOSC Bridge with a separate module. Config only needs a field for destination IP.

Reduce "Send string" to fixed OSC path "/sysex" and field "Value" becomes "SysEx string".

Reduce "Send message with multiple arguments" to fixed OSC path "/key", offer a pull down menu with all possible letters and special keys, offer 4 check boxes for modifiers. Offer radio button / pull down for "key down / key up".

Reduce "Send MIDI" to fixed OSC path "/midi" and fields for the 4 bytes. Offer same selectors for MIDI messages as midi-relay module.

That's it. So I would guess not too much hassle for an experienced module programmer as most things already have been done for generic OSC module and midi-realy.

:-)

For better understanding I attached a Companion config file. From what I understand it is necessary to disable/enable all modules to see the full button config.

Demo.zip

[XENONChromatic]

Things will be changing on the TouchOSC side of things soon, just as a general note. I am not the developer and cannot speak for them, but its something to be aware of.

I am going to loop in someone from his team to put this on their radar as they may have some additional comments to add.

[slvmchn]

Hi everyone,

Xenonheavyindustries reached out to me about this thread, so maybe I can help provide some details/insight. You likely already know most of these details but I'll cover them anyways. TouchOSC communicates with TouchOSC Bridge via OSC messages; TouchOSC Bridge then converts these to the appropriate MIDI messages, and sends them down the "TouchOSC Bridge" virtual MIDI port the TouchOSC Bridge creates. This works similarly in the reverse order - TouchOSC Bridge receives a MIDI message, and converts it to OSC to send to TouchOSC. On the user side, this communication/conversion is hidden and not really relevant to the user; they assign a MIDI message to a button, hit the button and MIDI goes out the TouchOSC Bridge. If TouchOSC Bridge receives the same message from the host software, the button in the TouchOSC layout will visibly trigger.

We only ever put out macOS/Windows binaries for the TouchOSC Bridge (and also never documented its functionality in detail), but a user did recreate it for Linux machines using Python: https://github.com/velolala/touchosc2midi This might not be relevant here, but may be worth looking at the code if there are any nuances of the TouchOSC Bridge you have questions about, as it is an accurate recreation of the TouchOSC Bridge (albeit in Python, but functionality-wise should be identical). Also while searching, there is a paper by Albert Graff titled "A TouchOSC MIDI Bridge for Linux" from 2014, that might have some helpful information as it breaks down some elements of the TouchOSC Bridge in detail: http://lac.linuxaudio.org/2014/papers/5.pdf Interestingly enough I found a presentation he gave at the time on archive.org: https://archive.org/details/LAC2014ATouchoscMidiBridgeForLinux

Regarding the future of TouchOSC - we have been working on a modernized version of TouchOSC that addresses most all of the limitations of the original TouchOSC, as well as greatly expands the features. This new software will have native compatibility with the TouchOSC Bridge, but additionally does not rely on it the way the original TouchOSC did; the new version will be able to create its own "TouchOSC" virtual MIDI port on the host computer so depending on the setup the TouchOSC Bridge will not necessarily be necessary (however, it will still be the main method of getting MIDI sent from mobile devices). So while mobile devices will still need to use the TouchOSC Bridge, the new software will run on desktop operating systems as well. So, for instance, you could run TouchOSC on a touchscreen monitor, and whatever software you want to send to on a separate monitor (on the same machine), and have TouchOSC send MIDI to the other software using the native TouchOSC virtual MIDI port, and not using TouchOSC Bridge at all. However, that's not to say that's how it has to be done; the user could opt to solely use the TouchOSC Bridge instead. This kind of "backwards compatibility" with the TouchOSC Bridge should easily allow users to port their layouts to the new software and continue functionality as it was prior if they want (the new version also includes a native method of importing layouts from the original version).

Another thing to note: key press triggers are most likely not going to be implemented in the new TouchOSC (at least natively). This functionality exists in the TouchOSC Bridge already and won't be removed, but it has basically been hinted at that passing keypress events to the operating system will likely be culled as a feature in general from OS's (it's a pretty vulnerable attack vector for malware). We've been grappling with a couple of bugs with this feature that seem to have become more apparent in more recent operating systems (especially macOS, causing button presses with combinations of modifier keys to not trigger properly). We're hesitant to implement/further a feature that may soon be deemed obsolete, so this is just kind of a head's up that ideally try not to rely too much on keypresses in layouts if you can avoid it (and the simpler the keypress the better, due to this modifier key bug).

Anyways forgive my rant but that should be a bit of detail on the TouchOSC Bridge, and a bit of insight into its role in the future alongside the new modernized version of TouchOSC. Please feel free to message me on here if you have any questions or need more discrete technical detail (again, I'm not involved directly in development so I may have to pass some questions on to Rob), although emailing me directly at support@hexler.net will ensure I see the messages quicker and you'll likely get a quicker response.

It's very humbling that over a decade after release people are still trying to implement TouchOSC into their setups with various software/hardware, and has also been a big motivator in developing the new version, specifically addressing the limitations of the original version. We're estimating release ~2 months from now, but that's a very soft estimate - we don't have a deadline to adhere to, but we're also pushing towards the last one-over of release-ready features.

Anyways that's my rant, which may or may not contain useful/relevant information, but figured I'd at least pop my head in and say hi and let you know you can contact us directly (ideally email support@hexler.net) if you have any questions while working on this module.

Cheers and thank you for your use and support of TouchOSC!

[RickDDD]

Hi guys, thank you for your thoughts and the information about the future of TouchOSC.

I have been curious what is going on under the hood of TouchOSC Bridge for quite a while. I gained some knowledge about network traffic and stuff in the last months which made it possible for me to reverse-engineer it now. Funny that I never came across the mentioned Python version, but actually reading all the information from the code probably would have taken me longer than it took me to find out myself.

Even funnier (although it does not make use of the OSC messages discussed here) is the paper and presentation by Albert Gräf as it turned out he worked and works at the university in my town, only 2km from where I live. Small world.

So what do I take from the information @slvmchn provided?

Nice that TouchOSC will come to desktop OS, even though quite some good alternatives on desktop OS have popped up while TouchOSC was stuck on mobile OS. Curious what the price tag will be like. I'm also happy that there will be progress in features as - while always being rock solid - TouchOSC always has been quite slow with this. What I'm sad about is that already the last update dropped older Android versions. For me TouchOSC is the number 1 app to give old tablets a new life or even buy an old one for 15 bucks and use it for TouchOSC as the app is not resource hungry at all. I really hope the last old-Android version will stay available for a long time.

But all this is not important here as my request does not need any version of TouchOSC, only the TouchOSC Bridge app on the desktop.

So the main points are: From what I understood TouchOSC Bridge will stay available and keep its functionality. I don't know about MacOS, but on Windows it has always worked and has been 100% reliable, including keystrokes. It just sits in the taskbar and works. So it should be usable and functional at least until the end of Windows 10 and a module would be of good use for years.

As mentioned for me it already works 100% using generic TCP-UDP and generic OSC and I am happy with it. With my request I just follow the Companion philosophy: If you think something is useful, do not rely on generic modules and that everyone knows how to achieve the functionality, but request a dedicated module so everyone can use it with ease :-)

[JTF4]

Hey there! I was browsing through the module requests and came across this.

Are you looking to use ToushOSC bridge as a way to just send MIDI and keystrokes to a remote computer from companion?

If you are, then I would have a look at Midi Relay and Vicreo Listener. They both have excellent support in companion and i use them pretty much daily.

Midi relay: https://github.com/josephdadams/midi-relay

Vicreo Listener: https://jeffreydavidsz.github.io/VICREO-Listener/

[RickDDD]

I'm looking for sending MIDI and keystrokes via one app that is already sitting on 100.000s of machines (TouchOSC bridge) instead of installing 2 more apps.

[JTF4]

Here's a link to the module development wiki. It contains everything you need to get started on creating modules. I would recommend that you start with the generic OSC module as a boiler plate.

Unfortunately since there are two apps that specialize in remote MIDI an keystrokes most developers will pass over this module request as it is both redundant and not as fully featured as MIDI relay and Vicreo Listener.

[RickDDD]

Thank you, but that was actually the first thing I read before posting here. It's just too complicated for me, I don't even understand the "Getting started" part. I don't know IF I would be able to learn it, but it would be hard for sure and way too much effort to make Companion spit out 3 simple network messages in the end.

Actually for ME it's fine to use Generic OSC module and a bit of Generic TCP-UDP module to send to TouchOSC Bridge, but on the Companion website everyone is encouraged to ask for new modules to make things available for easy use to everyone. So I thought I'd share what I found out while reverse-engineering the popular TouchOSC Bridge.