search

ossrs / srs

SRS is a simple, high-efficiency, real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181.
https://ossrs.io
MIT License
25.32k stars 5.33k forks source link

STREAM: [NFR] Can it support mixed configuration of multiple streams and ignore when the additional stream has no data? #721

Closed dreamsxin closed 4 years ago

dreamsxin commented 7 years ago

Generally used for audio stream synthesis or special effects.

TRANS_BY_GPT3

winlinvip commented 7 years ago

Mixing is quite complicated, we'll see if future versions can support it.

TRANS_BY_GPT3

dreamsxin commented 7 years ago

Or provide the cutting stream function, which can facilitate third-party tools for mixing and pushing. For example: streamOut serves as the external output stream, by default pulling the stream from streamIn. streamIn is the client input stream.

If a third party wants to mix StreamIn, they can first pull the stream from streamIn, then synthesize a new stream streamMux, and push it to streamOut. If this switching can be seamless, it would also be a solution.

TRANS_BY_GPT3

w5750584 commented 7 years ago

@dreamsxin May I ask, what third-party tool are you using for mixing? I tested using ffmpeg for mixing, but if one of the streams is interrupted, there will be issues with streamOut. ./ffmpeg -i rtmp://srs.xxx.com:1935/live/1 -i rtmp://srs.xxx.com:1935/live/2 -filter_complex "[0:a][1:a]amerge=inputs=2[aout]" -map "[aout]" -f flv rtmp://srs.xxx.com:1935/live/test When stream 1 or 2 is disconnected, the test stream will encounter an error.

TRANS_BY_GPT3

dreamsxin commented 7 years ago

@w5750584 Mixing, mixing in an audio file, specifying time and length, there will be no interruption problem, using a stream will definitely cause errors, unless you write it yourself

TRANS_BY_GPT3

winlinvip commented 7 years ago

I plan to mix the audio at the PCM level myself. SRS sends AAC or Speex or Opus frames to an external program, which decodes them into PCM, mixes them, encodes them back into AAC, and then sends them back to SRS.

TRANS_BY_GPT3

winlinvip commented 7 years ago

This program can be written in GOLANG, which makes it convenient to communicate with SRS using HTTP. Some work has already been done, including using GOLANG to call the libraries for SPEEX and AAC, as well as resampling.

  1. https://github.com/winlinvip/go-speex
  2. https://github.com/winlinvip/go-fdkaac
  3. https://github.com/winlinvip/go-aresample

TRANS_BY_GPT3

winlinvip commented 4 years ago

Currently, it seems that mixing may not be necessary, and possible directions could be:

  1. RTC communication direction, such as WebRTC, where the server typically does audio ranking instead of mixing. #307
  2. RTSP monitoring direction, such as GB18181, #1500, where there may be audio transcoding to AAC.
  3. Low-latency live streaming direction, such as WebRTC streaming, where Opus to AAC conversion may exist. #307

TRANS_BY_GPT3