MIDI Support

[wel97459]

I think it would be cool to have MIDI input and output inside Neos. So that you could make Neos into a type of VR MIDI controller. Or use a program like Ableton live to output MIDI to control lighting in a VR club.

[Frooxius]

Yeah I was thinking similar thing. One of the big features I plan on developing is SynthX, a framework for audio synthesis and filtering, so you could build a virtual studio, plugging things together, driving them from environment and other inputs and even physical hardware. Essentially so you could use Neos as a music studio.

Thanks! Will probably take some time to get to this one though.

[sirkitree]

Should we consolidate this into #69 ?

[Frooxius]

Probably not, they're somewhat different, although related features. MIDI files can be treated like just another audio file, whereas MIDI input is more about set of components for interfacing the inputs with objects and components in Neos.

[wel97459]

awesome)

[StarfishHidari]

this would be a super useful feature! currently we can interface with midi using websockets but having midi actually supported would be extremely helpful

[Toxic-Cookie]

SynthX sounds interesting. It should have it's own issue if there's not one already.

[StarfishHidari]

Yeah, it sounds really cool though it sounds separate and much larger than simple midi in/out, so it would do better in it's own issue rather than grouped into this

[Frooxius]

It's not fully separate issue, but rather a superset. It's a major pre-requisite to a lot of things like this, because MIDI input/output needs to naturally fit into that system and will be designed as part of it.

Otherwise if this was added and designed hastily, we might run into issue where it doesn't fit the rest of the audio processing system later on and we either break everyone's content or we severely restrict what we can do with audio.

[TehTurk]

Then to add on to the MIDI Support, if possible could we implement something like http://opensoundcontrol.org/ ? From what I'm aware it's a bit more of a modern implementation or network integration for MIDI as a whole and would be a boon to consider when MIDI support is added a bit more in full, some of the other bits and bobs of OSC could enable other things too :)

[StarfishHidari]

Official support for OSC would be incredibly nice, it's able to do an incredible array of things and great for networked use. It could handle things like lighting data, camera positioning/fov, avatar/object posing, etc.

As an example Virtual Motion Capture is built on top of OSC and used for a lot of vtubing applications, including allowing really good face tracking through iphone ARKit. I previously used a websocket for handling piping that into neos, but official support for OSC could make that process much simpler.

[jbaylies]

Check out https://alicelab.world/msvr/ ( https://github.com/worldmaking/mischmasch ) for inspiration.