search

react-native-webrtc / react-native-incall-manager

Handling media-routes/sensors/events during a audio/video chat on React Native
ISC License
555 stars 192 forks source link

Reenabling tracks causes lag in remote stream on iOS #226

Open gavrilikhin-d opened 1 year ago

gavrilikhin-d commented 1 year ago

When InCallManager is started in WebRTC call, it somehow creates lag on unmuting microphone on iOS. Without it everything works fine.

Unmuting with native logs ``` [javascript] setting muted to false (webrtc_voice_engine.cc:1486): Setting voice channel options: AudioOptions {} (webrtc_voice_engine.cc:424): WebRtcVoiceEngine::ApplyOptions: AudioOptions {audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, } (webrtc_voice_engine.cc:443): Always disable AEC on iOS. Use built-in instead. (webrtc_voice_engine.cc:453): Always disable AGC on iOS. Use built-in instead. (audio_device_generic.cc:18): BuiltInAECIsAvailable: Not supported on this platform (audio_device_generic.cc:28): BuiltInAGCIsAvailable: Not supported on this platform (audio_processing_impl.cc:911): AudioProcessing::ApplyConfig: AudioProcessing::Config{ pipeline: { maximum_internal_processing_rate: 48000, multi_channel_render: 0, multi_channel_capture: 0 }, pre_amplifier: { enabled: 0, fixed_gain_factor: 1 },capture_level_adjustment: { enabled: 0, pre_gain_factor: 1, post_gain_factor: 1, analog_mic_gain_emulation: { enabled: 0, initial_level: 255 }}, high_pass_filter: { enabled: 1 }, echo_canceller: { enabled: 0, mobile_mode: 0, enforce_high_pass_filtering: 1 }, noise_suppression: { enabled: 0, level: High }, transient_suppression: { enabled: 0 }, gain_controller1: { enabled: 0, mode: FixedDigital, target_level_dbfs: 3, compression_gain_db: 9, enable_limiter: 1, analog_gain_controller { enabled: 1, startup_min_volume: 0, clipped_level_min: 70, enable_digital_adaptive: 1, clipped_level_step: 15, clipped_ratio_threshold: 0.1, clipped_wait_frames: 300, clipping_predictor: { enabled: 0, mode: 0, window_length: 5, reference_window_length: 5, reference_window_delay: 5, clipping_threshold: -1, crest_factor_margin: 3, use_predicted_step: 1 }}}, gain_controller2: { enabled: 0, fixed_digital: { gain_db: 0 }, adaptive_digital: { enabled: 0, headroom_db: 6, max_gain_db: 30, initial_gain_db: 8, max_gain_change_db_per_second: 3, max_output_noise_level_dbfs: -50 }, input_volume_control : { enabled 0}} (webrtc_voice_engine.cc:1500): Set voice channel options. Current options: AudioOptions {audio_jitter_buffer_max_packets: 200, audio_jitter_buffer_fast_accelerate: 0, audio_jitter_buffer_min_delay_ms: 0, } Start Monitoring (RTCLogging.mm:33): (audio_device_ios.mm:444 OnGetPlayoutData): Possible playout audio glitch detected. Time since last OnGetPlayoutData was 123 ms. (RTCLogging.mm:33): (audio_device_ios.mm:453 OnGetPlayoutData): Glitch warning is ignored. Probably caused by device switch. (RTCLogging.mm:33): (RTCAudioSession.mm:855 -[RTCAudioSession observeValueForKeyPath:ofObject:change:context:]): OutputVolumeDidChange to 0.500000 (RTCLogging.mm:33): (RTCAudioSession.mm:531 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: (RTCLogging.mm:33): (RTCAudioSession.mm:544 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: CategoryChange to :AVAudioSessionCategoryMultiRoute (RTCLogging.mm:33): (RTCAudioSession.mm:563 -[RTCAudioSession handleRouteChangeNotification:]): Previous route: " ); outputs = ( "" )> Current route:" ); outputs = ( "" )> RNInCallManager.AudioRouteChange.Reason: CategoryChange. category=AVAudioSessionCategoryMultiRoute mode=AVAudioSessionModeDefault RNInCallManager.updateAudioRoute(): [Enter] forceSpeakerOn flag=0 media=video category=AVAudioSessionCategoryMultiRoute mode=AVAudioSessionModeDefault (RTCLogging.mm:33): (audio_device_ios.mm:630 HandleOutputVolumeChange): Output volume change detected. RNInCallManager.updateAudioRoute(): audioSession.overrideOutputAudioPort(.None) success (RTCLogging.mm:33): (audio_device_ios.mm:444 OnGetPlayoutData): Possible playout audio glitch detected. Time since last OnGetPlayoutData was 147 ms. (RTCLogging.mm:33): (audio_device_ios.mm:453 OnGetPlayoutData): Glitch warning is ignored. Probably caused by device switch. RNInCallManager.updateAudioRoute: audioSession.setCategory: AVAudioSessionCategoryPlayAndRecord, withOptions: 0 success RNInCallManager.updateAudioRoute() audio category has changed to AVAudioSessionCategoryPlayAndRecord (RTCLogging.mm:33): (audio_device_ios.mm:444 OnGetPlayoutData): Possible playout audio glitch detected. Time since last OnGetPlayoutData was 143 ms. (RTCLogging.mm:33): (audio_device_ios.mm:453 OnGetPlayoutData): Glitch warning is ignored. Probably caused by device switch. RNInCallManager.updateAudioRoute: audioSession.setMode(AVAudioSessionModeVideoChat) success RNInCallManager.updateAudioRoute() audio mode has changed to AVAudioSessionModeVideoChat RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: End RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: Unknow Value (RTCLogging.mm:33): (RTCAudioSession.mm:855 -[RTCAudioSession observeValueForKeyPath:ofObject:change:context:]): OutputVolumeDidChange to 0.750000 (RTCLogging.mm:33): (RTCAudioSession.mm:531 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: (RTCLogging.mm:33): (RTCAudioSession.mm:544 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: CategoryChange to :AVAudioSessionCategoryPlayAndRecord (RTCLogging.mm:33): (RTCAudioSession.mm:563 -[RTCAudioSession handleRouteChangeNotification:]): Previous route: " ); outputs = ( "" )> Current route:" ); outputs = ( "" )> RNInCallManager.AudioRouteChange.Reason: CategoryChange. category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat RNInCallManager.updateAudioRoute(): [Enter] forceSpeakerOn flag=0 media=video category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat RNInCallManager.updateAudioRoute(): audioSession.overrideOutputAudioPort(.None) success RNInCallManager.updateAudioRoute() did NOT change audio category RNInCallManager.updateAudioRoute() did NOT change audio mode RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: End RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: Unknow Value (RTCLogging.mm:33): (RTCAudioSession.mm:855 -[RTCAudioSession observeValueForKeyPath:ofObject:change:context:]): OutputVolumeDidChange to 0.900000 (RTCLogging.mm:33): (RTCAudioSession.mm:531 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: (RTCLogging.mm:33): (RTCAudioSession.mm:544 -[RTCAudioSession handleRouteChangeNotification:]): Audio route changed: CategoryChange to :AVAudioSessionCategoryPlayAndRecord (RTCLogging.mm:33): (RTCAudioSession.mm:563 -[RTCAudioSession handleRouteChangeNotification:]): Previous route: " ); outputs = ( "" )> Current route:" ); outputs = ( "" )> RNInCallManager.AudioRouteChange.Reason: CategoryChange. category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat RNInCallManager.updateAudioRoute(): [Enter] forceSpeakerOn flag=0 media=video category=AVAudioSessionCategoryPlayAndRecord mode=AVAudioSessionModeVideoChat RNInCallManager.updateAudioRoute(): audioSession.overrideOutputAudioPort(.None) success RNInCallManager.updateAudioRoute() did NOT change audio category RNInCallManager.updateAudioRoute() did NOT change audio mode RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: End RNInCallManager.AudioRouteChange.SilenceSecondaryAudioHint: Unknow Value (RTCLogging.mm:33): (audio_device_ios.mm:509 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): { category: AVAudioSessionCategoryMultiRoute categoryOptions: 0 mode: AVAudioSessionModeVideoChat isActive: 1 sampleRate: 48000.00 IOBufferDuration: 0.021333 outputNumberOfChannels: 2 inputNumberOfChannels: 2 outputLatency: 0.001458 inputLatency: 0.000000 outputVolume: 0.900000 } (RTCLogging.mm:33): (audio_device_ios.mm:520 HandleSampleRateChange): Handling sample rate change. (RTCLogging.mm:33): (audio_device_ios.mm:549 HandleSampleRateChange): Handling playout sample rate change: Session sample rate: 48000.000000 frames_per_buffer: 1024 ADM sample rate: 48000.000000 frames_per_buffer: 1024 (RTCLogging.mm:33): (audio_device_ios.mm:554 HandleSampleRateChange): Ignoring sample rate change since audio parameters are intact. (RTCLogging.mm:33): (audio_device_ios.mm:630 HandleOutputVolumeChange): Output volume change detected. (connection.cc:1441): Conn[1f5cd400:0:Net[en0:192.168.0.x/24:Wifi:id=1]:YlHZRgcg:1:0:local:udp:192.168.0.x:64990->sZf368Fw:1:2122260223:local:udp:169.254.163.x:57879|C--I|-|0|0|9115038255631187454|-]: Sent STUN BINDING request, id=5752563344375a6f63614f33, use_candidate=1, nomination=0 (RTCLogging.mm:33): (audio_device_ios.mm:509 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): { category: AVAudioSessionCategoryPlayAndRecord categoryOptions: 0 mode: AVAudioSessionModeVideoChat isActive: 1 sampleRate: 48000.00 IOBufferDuration: 0.021333 outputNumberOfChannels: 2 inputNumberOfChannels: 2 outputLatency: 0.001458 inputLatency: 0.000000 outputVolume: 0.900000 } (RTCLogging.mm:33): (audio_device_ios.mm:520 HandleSampleRateChange): Handling sample rate change. (RTCLogging.mm:33): (audio_device_ios.mm:549 HandleSampleRateChange): Handling playout sample rate change: Session sample rate: 48000.000000 frames_per_buffer: 1024 ADM sample rate: 48000.000000 frames_per_buffer: 1024 (RTCLogging.mm:33): (audio_device_ios.mm:554 HandleSampleRateChange): Ignoring sample rate change since audio parameters are intact. (RTCLogging.mm:33): (audio_device_ios.mm:630 HandleOutputVolumeChange): Output volume change detected. (RTCLogging.mm:33): (audio_device_ios.mm:509 HandleValidRouteChange): RTC_OBJC_TYPE(RTCAudioSession): { category: AVAudioSessionCategoryPlayAndRecord categoryOptions: 0 mode: AVAudioSessionModeVideoChat isActive: 1 sampleRate: 48000.00 IOBufferDuration: 0.021333 outputNumberOfChannels: 2 inputNumberOfChannels: 2 outputLatency: 0.001458 inputLatency: 0.000000 outputVolume: 0.900000 } (RTCLogging.mm:33): (audio_device_ios.mm:520 HandleSampleRateChange): Handling sample rate change. (RTCLogging.mm:33): (audio_device_ios.mm:549 HandleSampleRateChange): Handling playout sample rate change: Session sample rate: 48000.000000 frames_per_buffer: 1024 ADM sample rate: 48000.000000 frames_per_buffer: 1024 (RTCLogging.mm:33): (audio_device_ios.mm:554 HandleSampleRateChange): Ignoring sample rate change since audio parameters are intact. ```

Came from: https://github.com/react-native-webrtc/react-native-webrtc/issues/1424

gavrilikhin-d commented 1 year ago

This problem appears on latest version (4.1.0) too

ygorgasparin commented 3 months ago

Any updates on this issue, guys? Is there anything that can be done to work around it?