search

mpromonet / webrtc-streamer

WebRTC streamer for V4L2 capture devices, RTSP sources and Screen Capture
https://webrtcstreamer.agreeabletree-365b9a90.canadacentral.azurecontainerapps.io/?layout=2x2
The Unlicense
2.84k stars 584 forks source link

No audio stream from the streamer #458

Open ihabZhaika opened 2 years ago

ihabZhaika commented 2 years ago

Hey, I am feeding .mkv video with audio and I have a webcam with audio[there is microphone input], but in the streamer I get the player with the sound muted, when I checked the /api/getMediaList I get the following:

[
    {
        "video" : "OBS Virtual Camera"
    },
    {
        "video" : "hd_near_5mbs"
    }
]

When checking /api/getAudioDeviceList i get:

[
    "default: Built-in Audio Analog Stereo",
    "Built-in Audio Analog Stereo",
    "Remapped Monitor of Virtual-Speaker"
]

I have tried to add &%audio="Remapped Monitor of Virtual-Speaker" [encoded to url of course] but still the mute button is active and cannot be unmutd

In the streamer I have the following but it seems that it can recognize the audio : 

./webrtc-streamer-gcc -N 4 -n hd_near_5mbs -u file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv  -vvv
tester@WebRTC-NUC-1-Ihab:~/Downloads/streamer/webrtc-streamer$ ./webrtc-streamer-gcc -N 4 -n hd_near_5mbs -u file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv  -vvv
Version:v0.6.4-27-g73d1720-dirty/Linux-x86_64 civetweb@v1.13 webrtc@d91edc2244-dirty live555helper@d351689
{
        "urls" :
        {
                "hd_near_5mbs" :
                {
                        "video" : "file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv"
                }
        }
}Logger level:0
[000:000][265842] (field_trial.cc:140): Setting field trial string:WebRTC-FrameDropper/Disabled/
[000:000][265842] (audio_device_buffer.cc:64): AudioDeviceBuffer::ctor
[000:000][265842] (audio_device_impl.cc:136): current platform is Linux
[000:000][265842] (audio_device_impl.cc:155): CreatePlatformSpecificObjects
[000:000][265842] (audio_device_impl.cc:947): PlatformAudioLayer
[000:000][265842] (audio_device_impl.cc:266): PulseAudio support is enabled.
[000:000][265842] (audio_device_impl.cc:271): Linux PulseAudio APIs will be utilized
[000:000][265842] (audio_device_impl.cc:312): AttachAudioBuffer
[000:000][265842] (audio_device_buffer.cc:180): SetRecordingSampleRate(0)
[000:000][265842] (audio_device_buffer.cc:186): SetPlayoutSampleRate(0)
[000:000][265842] (audio_device_buffer.cc:200): SetRecordingChannels(0)
[000:000][265842] (audio_device_buffer.cc:206): SetPlayoutChannels(0)
[000:000][265842] (audio_device_impl.cc:332): Init
[000:001][265842] (audio_device_pulse_linux.cc:1571): mainloop running!
[002:027][265842] (audio_device_pulse_linux.cc:1305): context state cb
[002:027][265842] (audio_device_pulse_linux.cc:1315): no state
[002:027][265844] (audio_device_pulse_linux.cc:1305): context state cb
[002:027][265844] (audio_device_pulse_linux.cc:1315): no state
[002:027][265844] (audio_device_pulse_linux.cc:1305): context state cb
[002:027][265844] (audio_device_pulse_linux.cc:1315): no state
[002:028][265844] (audio_device_pulse_linux.cc:1305): context state cb
[002:028][265844] (audio_device_pulse_linux.cc:1324): ready
[002:028][265842] (audio_mixer_manager_pulse_linux.cc:86): the PulseAudio objects for the mixer has been set
[002:028][265842] (audio_device_pulse_linux.cc:1450): checking PulseAudio version: 13.99.1
[002:028][265842] (audio_processing_impl.cc:282): Injected APM submodules:
Echo control factory: 0
Echo detector: 0
Capture analyzer: 0
Capture post processor: 0
Render pre processor: 0
[002:028][265842] (audio_processing_impl.cc:291): Denormal disabler: supported
[002:028][265842] (webrtc_voice_engine.cc:321): WebRtcVoiceEngine::WebRtcVoiceEngine
[002:028][265842] (webrtc_voice_engine.cc:343): WebRtcVoiceEngine::Init
[002:028][265842] (webrtc_voice_engine.cc:351): Supported send codecs in order of preference:
[002:028][265842] (webrtc_voice_engine.cc:354): opus/48000/2 { minptime=10 useinbandfec=1 } (111)
[002:028][265842] (webrtc_voice_engine.cc:354): red/48000/2 { =111/111 } (63)
[002:028][265842] (webrtc_voice_engine.cc:354): ISAC/16000/1 (103)
[002:028][265842] (webrtc_voice_engine.cc:354): ISAC/32000/1 (104)
[002:028][265842] (webrtc_voice_engine.cc:354): G722/8000/1 (9)
[002:028][265842] (webrtc_voice_engine.cc:354): ILBC/8000/1 (102)
[002:028][265842] (webrtc_voice_engine.cc:354): PCMU/8000/1 (0)
[002:028][265842] (webrtc_voice_engine.cc:354): PCMA/8000/1 (8)
[002:028][265842] (webrtc_voice_engine.cc:354): CN/32000/1 (106)
[002:028][265842] (webrtc_voice_engine.cc:354): CN/16000/1 (105)
[002:028][265842] (webrtc_voice_engine.cc:354): CN/8000/1 (13)
[002:028][265842] (webrtc_voice_engine.cc:354): telephone-event/48000/1 (110)
[002:028][265842] (webrtc_voice_engine.cc:354): telephone-event/32000/1 (112)
[002:028][265842] (webrtc_voice_engine.cc:354): telephone-event/16000/1 (113)
[002:028][265842] (webrtc_voice_engine.cc:354): telephone-event/8000/1 (126)
[002:028][265842] (webrtc_voice_engine.cc:357): Supported recv codecs in order of preference:
[002:028][265842] (webrtc_voice_engine.cc:360): opus/48000/2 { minptime=10 useinbandfec=1 } (111)
[002:028][265842] (webrtc_voice_engine.cc:360): red/48000/2 { =111/111 } (63)
[002:028][265842] (webrtc_voice_engine.cc:360): ISAC/16000/1 (103)
[002:028][265842] (webrtc_voice_engine.cc:360): ISAC/32000/1 (104)
[002:028][265842] (webrtc_voice_engine.cc:360): G722/8000/1 (9)
[002:028][265842] (webrtc_voice_engine.cc:360): ILBC/8000/1 (102)
[002:028][265842] (webrtc_voice_engine.cc:360): PCMU/8000/1 (0)
[002:028][265842] (webrtc_voice_engine.cc:360): PCMA/8000/1 (8)
[002:028][265842] (webrtc_voice_engine.cc:360): CN/32000/1 (106)
[002:028][265842] (webrtc_voice_engine.cc:360): CN/16000/1 (105)
[002:028][265842] (webrtc_voice_engine.cc:360): CN/8000/1 (13)
[002:028][265842] (webrtc_voice_engine.cc:360): telephone-event/48000/1 (110)
[002:028][265842] (webrtc_voice_engine.cc:360): telephone-event/32000/1 (112)
[002:028][265842] (webrtc_voice_engine.cc:360): telephone-event/16000/1 (113)
[002:028][265842] (webrtc_voice_engine.cc:360): telephone-event/8000/1 (126)
[002:028][265842] (audio_device_impl.cc:332): Init
[002:028][265842] (audio_device_impl.cc:676): SetPlayoutDevice(0)
[002:028][265842] (audio_device_pulse_linux.cc:667): number of availiable output devices is 2
[002:028][265842] (audio_device_impl.cc:365): InitSpeaker
[002:029][265842] (audio_mixer_manager_pulse_linux.cc:146): AudioMixerManagerLinuxPulse::OpenSpeaker(deviceIndex=16)
[002:029][265842] (audio_mixer_manager_pulse_linux.cc:160): the output mixer device is now open
[002:029][265842] (audio_device_impl.cc:580): StereoPlayoutIsAvailable
[002:029][265842] (audio_device_impl.cc:587): output: 1
[002:029][265842] (audio_device_impl.cc:592): SetStereoPlayout(1)
[002:029][265842] (audio_device_buffer.cc:206): SetPlayoutChannels(2)
[002:029][265842] (audio_device_impl.cc:738): SetRecordingDevice(0)
[002:029][265842] (audio_device_pulse_linux.cc:788): number of availiable input devices is 2
[002:029][265842] (audio_device_impl.cc:371): InitMicrophone
[002:029][265842] (audio_mixer_manager_pulse_linux.cc:167): AudioMixerManagerLinuxPulse::OpenMicrophone(deviceIndex=19)
[002:029][265842] (audio_mixer_manager_pulse_linux.cc:182): the input mixer device is now open
[002:029][265842] (audio_device_impl.cc:534): StereoRecordingIsAvailable
[002:029][265842] (audio_mixer_manager_pulse_linux.cc:456): AudioMixerManagerLinuxPulse::StereoRecordingIsAvailable() => available=1
[002:029][265842] (audio_device_impl.cc:541): output: 1
[002:029][265842] (audio_device_impl.cc:546): SetStereoRecording(1)
[002:029][265842] (audio_device_buffer.cc:200): SetRecordingChannels(2)
[002:029][265842] (audio_device_impl.cc:851): RegisterAudioCallback
[002:029][265842] (webrtc_voice_engine.cc:439): WebRtcVoiceEngine::ApplyOptions: AudioOptions {aec: 1, agc: 1, ns: 1, hf: 1, swap: 0, audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, audio_jitter_buffer_enable_rtx_handling: 0, typing: 1, experimental_agc: 0, experimental_ns: 0, residual_echo_detector: 1, }
[002:029][265842] (audio_device_impl.cc:867): BuiltInAECIsAvailable
[002:029][265842] (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform
[002:029][265842] (audio_device_impl.cc:870): output: 0
[002:029][265842] (audio_device_impl.cc:883): BuiltInAGCIsAvailable
[002:029][265842] (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform
[002:029][265842] (audio_device_impl.cc:886): output: 0
[002:029][265842] (audio_device_impl.cc:899): BuiltInNSIsAvailable
[002:029][265842] (audio_device_generic.cc:38): BuiltInNSIsAvailable: Not supported on this platform
[002:029][265842] (audio_device_impl.cc:902): output: 0
[002:029][265842] (webrtc_voice_engine.cc:548): Stereo swapping enabled? 0
[002:029][265842] (webrtc_voice_engine.cc:553): NetEq capacity is 200
[002:029][265842] (webrtc_voice_engine.cc:559): NetEq fast mode? 0
[002:029][265842] (webrtc_voice_engine.cc:565): NetEq minimum delay is 0
[002:029][265842] (webrtc_voice_engine.cc:571): NetEq handle reordered packets? 0
[002:029][265842] (webrtc_voice_engine.cc:641): NS set to 1
[002:029][265842] (webrtc_voice_engine.cc:645): Typing detection is enabled? 1
[002:029][265842] (audio_processing_impl.cc:523): AudioProcessing::ApplyConfig: AudioProcessing::Config{ pipeline: { maximum_internal_processing_rate: 48000, multi_channel_render: 0, multi_channel_capture: 0 }, pre_amplifier: { enabled: 0, fixed_gain_factor: 1 },capture_level_adjustment: { enabled: 0, pre_gain_factor: 1, post_gain_factor: 1, analog_mic_gain_emulation: { enabled: 0, initial_level: 255 }}, high_pass_filter: { enabled: 1 }, echo_canceller: { enabled: 1, mobile_mode: 0, enforce_high_pass_filtering: 1 }, noise_suppression: { enabled: 1, level: High }, transient_suppression: { enabled: 0 }, voice_detection: { enabled: 1 }, gain_controller1: { enabled: 1, mode: AdaptiveAnalog, target_level_dbfs: 3, compression_gain_db: 9, enable_limiter: 1, analog_level_minimum: 0, analog_level_maximum: 255, analog_gain_controller { enabled: 1, startup_min_volume: 0, clipped_level_min: 70, enable_digital_adaptive: 1, clipped_level_step: 15, clipped_ratio_threshold: 0.1, clipped_wait_frames: 300, clipping_predictor:  { enabled: 0, mode: 0, window_length: 5, reference_window_length: 5, reference_window_delay: 5, clipping_threshold: -1, crest_factor_margin: 3, use_predicted_step: 1 }}}, gain_controller2: { enabled: 0, fixed_digital: { gain_db: 0 }, adaptive_digital: { enabled: 0, dry_run: 0, headroom_db: 6, max_gain_db: 30, initial_gain_db: 8, vad_reset_period_ms: 1500, adjacent_speech_frames_threshold: 12, max_gain_change_db_per_second: 3, max_output_noise_level_dbfs: -50}}, residual_echo_detector: { enabled: 1 }, level_estimation: { enabled: 0 }}
[002:029][265842] (render_delay_buffer.cc:378): Applying total delay of 5 blocks.
[002:029][265842] (matched_filter.cc:461): Filter 0: start: 0 ms, end: 128 ms.
[002:029][265842] (matched_filter.cc:461): Filter 1: start: 96 ms, end: 224 ms.
[002:029][265842] (matched_filter.cc:461): Filter 2: start: 192 ms, end: 320 ms.
[002:029][265842] (matched_filter.cc:461): Filter 3: start: 288 ms, end: 416 ms.
[002:029][265842] (matched_filter.cc:461): Filter 4: start: 384 ms, end: 512 ms.
[002:030][265842] (transparent_mode.cc:239): AEC3 Transparent Mode: Legacy
[002:030][265842] (echo_canceller3.cc:750): AEC3 created with sample rate: 16000 Hz, num render channels: 1, num capture channels: 1
[002:030][265842] (clipping_predictor.cc:357): [agc] Clipping prediction disabled.
[002:030][265842] (agc_manager_direct.cc:69): [agc] GetMinMicLevel
[002:030][265842] (agc_manager_direct.cc:73): [agc] Using default min mic level: 12
HTTP Listen at 0.0.0.0:8000

when I open the web page, I have the player muted[no option to unmute] image

For the "obs" webcam, I get the following

[018:074][279290] (CapturerFactory.h:275): audiourl:OBS Virtual Camera idx_audioDevice:-1/3
[018:074][279290] (PeerConnectionManager.cpp:1155): Cannot create capturer audio:OBS Virtual Camera

So am I missing something ?

ihabZhaika commented 2 years ago

Update: need to add the audio option and use the audio output name without the quotation in it

ihabZhaika commented 2 years ago

@mpromonet How is it possible to hear the audio of specific stream if I have used file:// to stream it ? since in the audio devices there is the general audio of the PC which is streaming

mpromonet commented 2 years ago

Hi,

Only few audio codec are supported, opus or pcm. To try it you could add an audio url.

                "hd_near_5mbs" :
                {
                        "audio" : "file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv",
                        "video" : "file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv"
                }

Best Regards, Michel.

ihabZhaika commented 2 years ago

Hi,

Only few audio codec are supported, opus or pcm. To try it you could add an audio url.

                "hd_near_5mbs" :
                {
                        "audio" : "file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv",
                        "video" : "file:///home/tester/Downloads/videos/mortal_shell_game_1080p_60fps.mkv"
                }

Best Regards, Michel.

Ok, Tried to separate them as shown in the JSON below, also tried to use the name of the file in the audio, did not work either

            "nfs":{
                    "video":"file:///home/tester/Downloads/videos/NFS_NO_AUDIO.mkv",
                    "audio":"file:///home/tester/Downloads/videos/nfs_sound.opus"
            }

The files are contained in this google drive if you want to give it a try, What else I can try ? and do you have a video that I can try it that do contain sound that will work with the streamer ?

adwpc commented 2 years ago

@ihabZhaika @mpromonet Hi, how to add audio device to stream, Thanks! i can see video only when running demo

ihabZhaika commented 2 years ago

@adwpc In my case to I were not able to watch multi streams where each one has it's sound track not even a single stream, tried to use different sound codecs as above but did not reach to a working solution, I do not know if I did something wrong buy if @mpromonet have some video with working audio so we can try it would be great

hantconny commented 2 years ago

ffmpeg -re -stream_loop -1 -i /home/videos/1080p_cam3.mp4 -c:v libx264 -c:a opus -strict -2 -f rtsp rtsp://127.0.0.1/live3

docker run --network host --name webrtc-streamer -it mpromonet/webrtc-streamer -v -H 8000 -n test -u rtsp://127.0.0.1/live3 -U rtsp://127.0.0.1/live3

it works