P2P streaming

[RussCoty]

Idea

I'm on lockdown at the moment and I am trying to find a decent solution for the issues that people in my industry (audio/music) are encountering.

Applications for non-audio industry users would also be possible I am sure but right now I can see my industry in real trouble, not so much producers like myself who can afford the proprietary systems but musicians and small labels will suffer greatly. However this is already having a knock on effect on producers too. :(

Essentially what I want to create is some kind of open source solution for streaming full resolution audio/lossless audio to clients/collaborators.

There are a couple of systems out there already but they have certain issues, cost, 3rd party servers, only work with one specific audio workstation software etc.

I feel like that it would be viable to implement a system that would allow streaming collaboration for music/audio creators without these 3rd parties involved.

WHAT? Example: Someone is mixing a track in their home studio, client needs to listen in at full quality. Chat can be conducted via one of the many 3rd party applications....skype, FT, Zoom, etc. Mix engineer would drop a plug-in onto the main output feed for the audio and client would listen wither via an application or a website, probably a webserver hosted on the mix engineer's computer.

Another Example: I am producing a band who are all living in a flat together, they can record but they need someone to guide them through performances, mic positions etc. They drop the plug-in onto their maaster output feed and I listen to the high quality audio they are recording, we chat over Facetime on our phones.

Features: Part 1: Audio Plug-in to capture the audio feed from any DAW (digital audio workstation) and feed it to the streaming software. Maybe ffmpeg libs can be included here.

Some kind of P2P handshaing/code that could be delivered from the plugin/application to allow the client

Potentially Part 2: A file sharing system that could synchronise a folder full of project and audio files between multiple users online. A CRON job could compress recorded WAV files into a sub folder, these could be synced quickly and then expanded at the other end. Then a higher quality wav file could be synced. by doing it this way no alteration to the DAW software would be needed as the DAW doesn't realise that it is reading a temporary placeholder WAV file whlst the full quality file arrives in the background.

Platforms: I am currently learning JUCE a framework for C++ that allows audio plug-in development. It is essentially C++ with a bunch of libraries and a rather nice project manager package that dumps files into XCode for me. JUCE projects can compile to WIN/Linux/OSX Desktop, iOS and audio plug-in formats.

Where I am up to so far: I have been playing with FFMPEG and managed to stream audio and video to youtube, plus recording files seems quire simple. Streaming stereo at 48KHz 24 bit shouldn't be that impossible really.

There is a fair bit of info out there on P2P connections via FFMPEG that seems to be useful. However I have read that FFServer (now discontinued development) might be needed to allow multiple recipients. Not sure how this would all work.

Any suggestions/help/experience would be GREATLY APPRECIATED.

Questions:

Should the plug-in just feed a virtual soundcard like soundflower and that can then feed FFMPEG? Ths would actaully then be something similar to OBS but maybe with an audio focus.

Can the plug-in directly administer handshaking etc?

Am I crazy?

Thank severyone.

[amcewen]

Not sure I've got any great insight yet, but just noting that I've been working with someone recently who's done a fair bit of JUCE programming, IIRC, if you hit problems on that side.

Does WebRTC have options for different codecs or high-quality options? That's probably where I'd look first, along with things like ICEcast, as surely this is something people have already prodded in the past?

[RussCoty]

Thanks @amcewen I’ll look into that.

[MethodDan]

I used to do a lot of online radio stuff with ICEcast which works well for one way audio streaming. It's not P2P though. You need a server set up but it's all free and open source. For high quality audio you can set it to do 320kbps MP3 (still compressed I know) but it might also do FLAC which would be better. Most VoIP protocols weren't high enough quality for decent music I found but times may have changed.

For collaborating with artists or other musicians online in real time there used to be a project called NinJam, I never got a chance to try it properly but it used a timecode based solution and everyone playing a bar behind real time to allow jamming with people around the globe. That was the theory anyway, never got to see if it worked properly personally. I found the idea of playing and thinking a bar behind a bit odd but maybe I could have gotten used to it.

Looks like it has a plugin for reaper, dunno if it's a VST though - https://www.cockos.com/ninjam/

[dfaligertwood]

So the traditional method would probably be ISDN or similar, and there's a number of commercial options for this.. Source connect - source-elements.com Source Connect Now - now.source-elements.com SessionWire - www.sessionwire.com ConnectionOpen - www.connectionopen.com ipDTL cleanfeed.net Sessionlink Bodalgo Call

• Interestingly, a lot of these seem to be based around google chrome and ogg vorbis, which makes me think they're basically just using WebRTC. This would make sense, because it lets google do all the hard work of optimising quality and latency...

[MatthewCroughan]

Having set up Jitsi recently, you can just embed an instance of it inside of an iframe and have the server configured to facilitate p2p connections. Jitsi is very much architected to be a component of a larger system

Proof: This is how Element.io does video chats It is also how Veertly implements theirs.

Let me know how you want to try this @RussCoty