search

phoboslab / jsmpeg-vnc

A low latency, high framerate screen sharing server for Windows and client for browsers
2.39k stars 397 forks source link

Sound? #1

Open benjamingr opened 9 years ago

benjamingr commented 9 years ago

Is sound coming?

moughouz commented 9 years ago

Adding sound is not that hard, there will be only one downside and that is when the data is being transmitted. This could create a high latency(especially for low bandwidth users), but i don't think it will.

phoboslab commented 9 years ago

Capturing, encoding and transmitting sound is not that difficult, however, decoding and playing Sound in the browser is. Afaik there's currently no native Browser API that can deal with streaming audio, so we'd need to use a JavaScript decoder for that as well.

benjamingr commented 9 years ago

It's not standard yet, but in Firefox and Chrome you have https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_API

espadrine commented 9 years ago

Web Audio API is actually well supported and gives good latencies (around .5ms on desktop, 13ms on mobile).

Additionally, there already is a JS MP3 decoder that sends data to Web Audio API: https://github.com/audiocogs/mp3.js.

WebRTC (MediaStream) would be ideal, both for audio and video, as it is meant for low-latency playback (going as far as using SCTP over UDP instead of TCP). It may require a lot more work, though.

jochuan commented 8 years ago

any API that you can use ?

ghost commented 6 years ago

In the server side, what will be best api to capture the audio? any samples?