search

twilio / video-quickstart-ios

Twilio Video Quickstart for iOS
https://www.twilio.com/docs/api/video
MIT License
457 stars 177 forks source link

Example of mixing the sampleBuffer of appAudio provided by RPScreenRecorder or BroadcastExtenion. #595

Open Fanpei920 opened 3 years ago

Fanpei920 commented 3 years ago

Description

I'm doing on a project which has a feature that sharing the in-app webview sound through RPScreenRecorder.

Already read the source code of ExampleReplayKitAudioCapturer from ReplayKit example, AudioDeviceExample and implemented the custom audio device, but didn't go well, It crashes and the output sounds in a higher tone, Can you any example of mixing the sampleBuffer of appAudio provided by RPScreenRecorder or BroadcastExtenion?

Steps to Reproduce

  1. Launch webview view.
  2. Let webview play some sound (like Youtube)
  3. Share the screen Use RPScreenRecorder (or BroadcastExtenion)

Expected Behavior

The appAudio should be mixed into mic recorded audio, then share with the participant.

Implemented Architecture

Screen Shot 2021-03-22 at 11 27 57

Problems

Cannot correctly implement the custom device to mediate mic input and sampleBuffer provided from RPScreenRecorder.

Discussion

Partially duplicated with #519, but in order to clarify the problem I made this issue.

timrozum commented 3 years ago

HI @Fanpei920,

We have not implemented this ourselves yet with ReplayKit but it is on our roadmap. Unfortunately until then I don't think I can provide good insight into this.