Voice Echo on the same device where he spoke

[mdflores]

Description

During a call, when user speaks about 1.5s there would be a playback where you can hear what the user uttered. Note that the participants of the call are on different locations.

Steps to Reproduce

1. iOS Client 1 Connects to call via TwilioVideo SDK
2. iOS Client 2 Connects to call via TwilioVideo SDK
3. iOS Client 1 speaks to start.
4. Echo back from what the user spoke --> point of problem

Expected Behavior

During call, minimal or at least no echo when speaking.

Actual Behavior

During a call, when I speak there is an echo playing back what I have uttered

Reproduces How Often

100% Reproduced

Versions

All relevant version information for the issue.

Video iOS SDK

TwilioVideo (1.3.10) via CocoaPods

Xcode

Xcode 9.4

iOS Version

11.0.3

iOS Device

iPhone 6, iPhone 6 Plus

[andyboyd]

I'm seeing this same issue with the 2.1.0 SDK. Interestingly, I have a host/viewer setup using a replaykit extension, with the host (i.e. the extension) using a custom audio unit, and that side of the conversation does not experience the echo.

Plugging headphones in does eliminate the issue, so I suspect it's the echo cancellation on the TVIVideoView's audio unit not working correctly. Is TVIVideoView using a VoiceProcessingIO Audio Unit?

[mdflores]

It is still observed with the used of headphones.TVIVideoView is the default no custom audio unit.

[piyushtank]

Thanks @mdflores and @andyboyd. We have introduced custom audio device feature in version 2.x. When a custom device is not used in 2.x, TVIDefaultAudioDevice is used. The TVIDefaultAudioDevice uses VoiceProcessingIO and uses the in-built hardware echo cancellation. This is same in video sdk version 1.x also. i.e. in version 1.x the TVIAudioController uses VoiceProcessingIO and uses the in-built hardware echo cancellation.

We will try to reproduce the problem and get back. @mdflores I am assuming the problem is reproducible only in iPhone 6 and iPhone 6 plus running iOS 11.0.3 but not on other iPhone / iOS combinations. Please correct me if my understanding is not correct here.

Also, the video sdk 2.x SDK is more stable and have more features in it. I recommend using it in your app.

[mdflores]

Those are the only test devices I have currently, and not sure if this will happen on other devices. I will try also to move to 2.x

[piyushtank]

Thanks @mdflores I have created an escalation to reproduce this problem in our lab. I will post an update with my findings soon.

[piyushtank]

@mdflores We tried few times on iPhone 6 Plus running iOS 11.0.3 using Quickstart and our internal video app but we could not reproduce the problem. In order to debug this problem further, is it possible for you to share the code with us?

[mdflores]

Thank you for doing this.. Below is the code I used.

Creation Of Audio and Video Feeds

private func prepareMediaFeed() {
        self.serviceQueue?.async {
            if (self.localAudioTrack == nil) {
                self.localAudioTrack = TVILocalAudioTrack(options: nil, enabled: true)
                if (self.localAudioTrack == nil) {
                    print("Failed to create audio track")
                }
            }
            self.createVideoTrack()
        }
    }
private func createVideoTrack() {
        if UIDevice.current.isSimulator {
            return
        }
        if (self.localVideoTrack == nil) {
            let videoConstraints = TVIVideoConstraints { (constraints) in
                constraints.aspectRatio = TVIAspectRatio16x9
                constraints.maxFrameRate = 30
                constraints.minFrameRate = TVIVideoConstraintsFrameRateNone
            }
            self.camera = TVICameraCapturer(source: .frontCamera, delegate: nil)
            self.localVideoTrack = TVILocalVideoTrack(capturer: self.camera!, enabled: true, constraints: videoConstraints)
        }
    }

Connecting to Room (uses self.localVideoTrack and self.localAudioTrack)

private func connect(include videoTrack: TVILocalVideoTrack?, and audioTrack: TVILocalAudioTrack?) {

        self.serviceQueue?.async {
            guard let roomSettings = self.roomSettings else {
                return
            }

            let audioTracks = audioTrack != nil ? [audioTrack!] : [TVILocalAudioTrack]()
            let videoTracks = videoTrack != nil ? [videoTrack!] : [TVILocalVideoTrack]()

            let connectOptions = TVIConnectOptions(token: roomSettings.token) { (builder) in

                builder.roomName = roomSettings.roomId
                builder.videoTracks = videoTracks
                builder.audioTracks = audioTracks
            }
            self.room = TwilioVideo.connect(with: connectOptions, delegate: self)
            self.localParticipant = self.room?.localParticipant
        }
    }

[mdflores]

ooopS! Reopening.. My Bad Wrong button

[piyushtank]

@mdflores The code snippet you have shared looks OK to me, however, without looking at the full, hard to say.

Given that we are not able to reproduce the problem locally on the exact device+os combination, are you able to reproduce the problem using our Quickstart sample app?

[piyushtank]

@mdflores following up on the last post. Are you still observing the problem? If yes, since we are not able to reproduce the problem locally, can you try our Quickstart sample app and let us know? As mention earlier, the video sdk uses in-built hardware echo cancellation. However, in future releases, we are planning to provide and API to support software echo cancellation in case if inbuilt echo cancellation does not work.

[mdflores]

Hi apologies for the delay on the response. We found out that the receiver (Nexus 7) was causing the issue and not the iOS one. Thanks for your responses.