search

jameshball / osci-render

〰📺🔊 Synthesiser audio plugin for making music by drawing objects on an oscilloscope using audio.
https://osci-render.com/
GNU General Public License v3.0
428 stars 22 forks source link

Add MIDI/audio hook for DAWs #110

Closed DJLevel3 closed 10 months ago

DJLevel3 commented 2 years ago

Add a hook/plugin for DAWs

Issue

There is no way to use osci-render with a DAW such as Ableton or FL Studio at the moment, so recording audio with MIDI input is only possible when doing it live or with special software.

Potential solutions

  1. Add functionality to play back MIDI files! This one's probably quite easy and in my opinion should be implemented ASAP.
  2. Create a VST3 plugin using iPlug2 that just sends MIDI to the standalone osci-render application (like the Blender plugin sends GPLA) and returns audio taken from the application. This can likely be done using a web interface or using local IP addressing, as iPlug2 has a web interface functionality.
  3. Port osci-render into a VST3 plugin to run entirely within the DAW. This is probably very difficult, as iPlug2 and JUCE both use C++ rather than Java. While an interface plugin would only need C++ for the plugin side (which I can develop), a true port would mean converting everything to C++, which would take far too much time and effort.

Notes/Miscellaneous

If you want to go with option 1, I can develop the VST3 plugin, since I mainly program in C++, but I'd need someone else to do the osci-render part as I suck at Java.

Thanks for making such an amazing tool, by the way! I use it a lot for making music, because for most use cases it's the best tool available.

Edits

8/24/2022 @ 11:19 AM Mountain Time: Added another potential solution

DJLevel3 commented 2 years ago

Update! I think I can actually do it using a standard MIDI connection and a TCP socket to return the audio to the VST plugin. I can probably even do it all myself since the Java documentation is quite good.

jameshball commented 2 years ago

I've been trying to get something like this working for ages and didn't even know your 1st solution was an option! If you think that's possible then it would be amazing if you could look into this. Let me know if there's anything at all I can help with on the Java side of things or if you have any questions :)

Happy for you to lead this though as you have the most experience with using DAWs!

DJLevel3 commented 2 years ago

Thanks! I realized two things just now: First, MIDI is dead simple to send over a TCP connection. Second, MIDI file playback is an easy to implement feature that would fit most applications. I'm looking into implementation right now, but if all goes well, something will be implemented later today.

jameshball commented 2 years ago

Amazing, thanks again!

DJLevel3 commented 2 years ago

With some external software, it's possible to get MIDI out of a DAW with no plugin! The one issue is latency and performance, it was super finicky to get it to work right and it's definitely not live. There's a significant amount of buffering time. However, you can still get MIDI from a DAW to osci-render and then record the audio from OR, like I did here!

https://youtube.com/shorts/j5qOVXVcDoQ?feature=share

jameshball commented 2 years ago

This is really cool and a great start :) where do you think the latency is coming from in this case? Is it DAW-specific or not?

DJLevel3 commented 2 years ago

It's the MIDI loopback software I used. osci-render doesn't show up as a MIDI device by default so I had to use a pipe to get the MIDI from Ableton to OR.

DJLevel3 commented 2 years ago

So if you add the functionality to somehow make OR show up as a device then this would likely work

jameshball commented 2 years ago

I use a loopback software called LoopBe - is this the one you're using? I didn't notice any latency issues with it in the past. From a quick look it doesn't look like there's a way of getting Java to register a MIDI device that the OS can see.

MIDI files in osci-render are definitely a sensible way to do this in a way that probably won't be that hard but wrapping this in a VST might be the best way to do this

DJLevel3 commented 2 years ago

I was using loopMIDI. I'll try LoopBe later today if I remember. https://www.tobias-erichsen.de/software/loopmidi.html

DJLevel3 commented 2 years ago

Huge difference! It's basically realtime, though my system hates it if I pipe the audio back into Ableton, haha

jameshball commented 2 years ago

That's a shame!! Yes. It gets upset when that happens with osci-render too lol

xatarox commented 2 years ago

Hey guys, I didn't read all of this but there is a simple solution to redirect your audio to your favorite DAW, just use VB (virtual audio cable) and redirect the output audio to the daw :)

DJLevel3 commented 2 years ago

The issue there is the performance overhead. It'd perform better and be easier to use if it was DAW-integrated.

jameshball commented 2 years ago

I'm going to tack on another point here about MIDI performance - currently tracking a few issues regarding the performance of osci-render whilst using MIDI and Lua simultaneously. I haven't investigated it yet, but I'm aware it's a problem!

jameshball commented 1 year ago

@DJLevel3 I'm currently working on a JUCE C++ port over at https://github.com/jameshball/osci-render-juce

It currently supports .obj files and has a very basic version of the sliders working. Will be a slow process porting stuff but a lot of the fundamental stuff is currently implemented. Will next be implementing Chinese postman to sort the lines being drawn, and then SVG parsing.

jameshball commented 10 months ago

The JUCE C++ port is now complete, so you can use osci-render as an audio plugin in your favourite DAW :) You can get it here!