Add support for webRTC

[toots]

There seems to be a good lib for that there: https://code.google.com/p/webrtc

[smimram]

This might take quite long to bind. So, for a starter I think it would be much more reasonable to find or program a small webRTC client using this lib, and use it in Liquidsoap as external.

[smimram]

Probably a good starting point is http://www.webrtc.org/reference/getting-started

[toots]

In the case of APIs with obviously a zillion of functions we will never need in liquidsoap, I believe that it's appropriate to start with a internal binding in liquidsoap, only covering what we need for input.webrtc and output.webrtc...

[smimram]

Even better: https://github.com/muaz-khan/WebRTC-Experiment/tree/master/RecordRTC

There is a very nice demo here: https://webrtc-experiment.appspot.com/RecordRTC/

It seems to me that it's only a matter of little javascript coding in order to be able to stream to an harbour from a browser!...

@toots: what do you think?

[smimram]

So, a quick plan could be to use

• web audio API for capturing audio: https://webrtc-experiment.appspot.com/RecordRTC/
• lame in js for encoding in mp3: https://github.com/akrennmair/libmp3lame-js
• websockets for communicating with harbor: http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-75

Basically, most of the work resides in having harbor follow the websocket protocol (which does not seem to be overly complicated).

[virtualfunction]

I'm going to chip in here too as I'd like to keep my eye on this.

I'm wondering about the performance aspect of using WebSockets and Harbor / Icecast based API as this means you'll need to decode in the browser and deal with the raw audio interfaces (of which isn't Firefox's interface getting changed?). Using a JS implementation of LAME I would assume isn't too efficient on the CPU (not that it matters too much for now) but further more a lot of people are adverse to the MP3 format (patents, high latency, no DTX). The other side of it is that Firefox and Chrome support Opus (and VP8) out of the box which is one of the key advantages of the Web RTC stack over competing tech such as Flash. My impression is that WebSockets is suited more for message based communication rather than streaming.

I'm not expert in this field, but my impression is that for media transport WebRTC is using (S)RTP over UDP with Ice/STUN/TURN/SDP for channel/transport negotiation. Based on this, I would have thought the best way forward here would be to implement something like output.rtp which could stream to dedicated server side software that already supports RTP such as Asterisk or Wowza (I would also imagine, and hope the Node JS community quickly picks up on this and provides some nice RTP libraries here given this would be far more controllable / lightweight than fudging and kludging some Asterisk AGI scripts or Java code for Wowza). It's then down to the server side software to deal with the streaming to the client, and this would probably save you needing to implement the negotiation which to me is outside of the remit of Liquidsoap. (Is this what you mean smimram by using Liquidsoap as an external?)

[smimram]

Yes, for websockets I agree with you, WebRTC support would be far better. I was just looking for a quick & dirty solution, but since it seems that it is not going to be quick and it is going to be quite dirty, I'm not so sure it is a good idea...

Concerning WebRTC, full support in Liquidsoap would be quite complicated, we should use some program with support for RTP to do the job as you suggest. So, the situation is: if you (or anybody) can come up with a program (in whichever language, using whichever libs / other programs) which can receive data for WebRTC and output it in some way (such as a wav file, or better on stdout), we'd be glad to try to integrate that to Liquidsoap!

[vrtisworks]

Since I was the one who opened this can of worms. Maybe I should apologize and ask the question a little differently since it seems complicated to do WebRTC.

I am writing browser based DJ application Pretty much a simple create playlists, feed the next song to LiquidSoap. The music will all be on the server with LiquidSoap, so I do not need to worry about sending that over. What I do not have a good way of doing is getting the DJ voice from the browser to LiquidSoap. Once I get that, the DJ can be anywhere, and there is zero client software to install.

I asked about WebRTC because that is what it seems people are using to do browser to browser voice.

I do not know if this makes it easier or not. But I thought I would clarify what I was trying to do.

Thanks for at least looking into it guys.

[smimram]

I understand your concern about zero-install for DJs, it would indeed be very nice to have. But there is unfortunately nothing ready out of the box for now :/ WebRTC would be a very good way to do this, but it is not simple to achieve support for this so it might take time. In the meantime, if you come accross some ideas about how we could do this (for instance a js library which is able to stream the mic in some way) please share them with us so that we can try to come up with a solution...

[toots]

I've had a more detailed look at webRTC's API yesterday, It is unfortunately very much web-oriented. First, signaling channel is not specified and is usually implemented by means of web technologies, such as WebSockets. Second, it is really oriented toward client-to-client communications.

I don't thing we'll have this working in liquidsoap any time soon. More concerning, I don't see webRTC as being of any easy implementation in the case of server-to-client communication, which is really unfortunate.

[vrtisworks]

Thanks for the time you guys took looking into this. Probably should just close the request.

I will continue my quest to figure out how to get the stream of bits from the microphone delivered to LiquidSoap. I suspect that I will need to add NodeJS into the mix. That should take care of getting the bits from the browser to the server. Now to just figure out how to get them from there to you. I'll be back with questions when I get closer.

Thanks again.

[virtualfunction]

Just a possible suggestion/route to investigate...

What about using Flash and red5? You can then use the red5 client libraries or anything that supports RTMP library to capture the steam from red5 and feed this to something like JACK. There is Java libraries for both of these.

Another option, which is probably easier and better (purely because its more open that Adobe's horrid Flash tech) is to stream up using Icecast by using a Java client, and then hooking liquidsoap up to Icecast on the server using input.http or input.harbor. Writing an Icecast client in Java shouldn't be too hard, you basically open a HTTP 1.0 connection using the source verb, send all the meta data as HTTP headers and encode your audio and stream up as your http body payload "as is". Best way to see how this works is the use Wireshark/tshark to capture the streaming process when using ices or liquidsoap as a source client.

[virtualfunction]

If you can get your head round the syntax/dialect of "Lots of Irritating Superfluous Parentheses" know as Clojure take a look at https://github.com/jblomo/iceclient

(This should compile to something the Java VM understands, so it should be possible to wrap this up behind java.applet.Applet - I'm just not sure how many hurdles Java will give you due to Applets running in an unprivileged mode in the browser)

[mcfiredrill]

Hi, I tried out the websockets branch and the test server to send the mp3 stream from the browser. Seemed to work fine but it didn't look like anything was added to the actual harbor code to support this yet. Are you still working on this or are you abandoning this approach in favor of something else? Also saw this repo: https://github.com/savonet/webcast

Very excited about the possibility of streaming from browser to harbor! :star: :star2: :stars:

[smimram]

Nothing is really commited yet, but stay tuned, websocket support in harbor is on his way! :)

[toots]

This one is a big no no for me now.. ^^

[Keyne]

How is this on nowadays? Is it possible to do it using gstreamer? (https://github.com/centricular/gstwebrtc-demos) Liquidsoap is already integrated with gstreamer. And it's already possible do have a RTP input, although I've not found a way to create the answer for the webrtc offer after loading the SDP.

[smimram]

Yes, apparently support has recently been integrated in GStreamer : http://blog.nirbheek.in/2018/02/gstreamer-webrtc.html

Let's reopen this bug :)

[romansavrulin]

Any news on this?

[smimram]

Nope. However, we're now moving toward ffmpeg. If you manage to get something working with ffmpeg command line, chances are high that we can have it quite quickly in Liquidsoap.

[romansavrulin]

Nice move! Will try to get it working!

[smimram]

Great, please keep us posted!

[toots]

Don't hesitate to reach out in our slack channel if you need!

[smimram]

Some notes for later

• it seems that ffmpeg does not natively handles webRTC
• however GStreamer does handle webRTC
• a good option might be the Janus server

There are a few libraries implementing webRTC:

• in C
• in go
• in python

Concerning webRTC:

• there are lots of demos here

[danrossi]

I'll add to this open ticket. Can this support whepsrc / webRTC via gstreamer ? The playout is required to go to the audiosink on a PI for playout. So playback client rather than restream