Recording audio too - Githubissues

chrisronline commented 5 years ago

What are you thoughts on adding (optional) audio recording too? I'd like to record the screen and the mic for my use case. The code looks fairly straight-forward and I got it working locally, but unsure if it should be on my default or perhaps I should publish a fork with the changes instead?

Thanks!

code-matt commented 5 years ago

I agree, someone had started a fork for in app audio. I pulled in the changes and started the record_inapp_audio_merge branch. I added recording microphone audio as well and fixed one crash. The reason I have not merged that branch in is it has changes for 60.0/iOS13 as well and waiting for it to release to fix the multitasking ReplayKit bug.

It would be great if you could take a look about how both audio streams are implemented on that branch. Feel free to make PR for any tweaks or suggestions. One thing I have not been able to get too is, both in app audio and microphone audio should be options passed in the initial config.

chrisronline commented 5 years ago

@code-matt When originally testing that branch, did you experience inconsistency with the microphone recording? The first couple times it worked (recorded the microphone audio successfully), but now it doesn't work (even though the code is unchanged to the best of my knowledge) at all. The symptom is in the startCapture callback, the bufferType is never audioMic even though the app has microphone permissions.

Any thoughts on this?

BTW, I am testing this on an actual iOS device (ios 12.4)

code-matt commented 5 years ago

@chrisronline I tried earlier and have not run into that issue. I went ahead printed the bufferType as well, always see all 3. The video writing only once stop is pressed. I tried breaking it, killing the app during a recording and other things and no luck.

I am experiencing a different one though, the in app audio being turned down to the point where can barely be heard during the actual period of recording. It's mixed fine with the microphone when playing back after. This seems to be the default behavior of iOS after looking into it though.

code-matt commented 5 years ago

@chrisronline I managed one worse after continuing for a while longer. I can no longer record anything after it working fine for nearly 30 minutes.

I do not see errors but when going to play back, always see:

[Generic] Video /private/var/mobile/Containers/Data/Application/7619D1C0-8E26-4C01-BF5E-D46AAB022C38/Documents/Replays/Recording-1567730454.mp4 cannot be saved to the photo library: Error Domain=AVFoundationErrorDomain Code=-11829 "Cannot Open" UserInfo={NSUnderlyingError=0x282bfe8e0 {Error Domain=NSOSStatusErrorDomain Code=-12848 "(null)"}, NSLocalizedFailureReason=This media may be damaged., NSURL=file:///private/var/mobile/Containers/Data/Application/7619D1C0-8E26-4C01-BF5E-D46AAB022C38/Documents/Replays/Recording-1567730454.mp4, NSLocalizedDescription=Cannot Open}

Even after uninstalling the app and restarting the device. Does not matter if I pick screen only or screen and microphone.

rwinright commented 5 years ago

Sorry I'm a little late to the party here, but I've been testing the branch with the audio implementation. It seems to work perfectly within the test app, but when I use it in my own app, it seems to not only corrupt the media files on stop, but I also get an error message in the stopRecording function that completely nukes the app saying "Thread 21: Fatal error: Unexpectedly found nil while implicitly unwrapping an Optional value".

code-matt commented 5 years ago

I have not been able to get anywhere with this, my initial findings were around needing to queue the audio/video writes. Have not been able to get back to it, though. I will leave the audio branch and prepare one for iOS13 without inapp or mic audio until this can be figured out.

code-matt commented 5 years ago

I pushed up a small change that fixed the one big crash for me. Just messing around for a long while with AVFoundationErrorDomain Code=-11823 eventually lead me to realize the session being started more than once by different buffers, I think.

                    if self.assetWriter.status == AVAssetWriter.Status.unknown && !self.sessionStarted
                    {
                        self.sessionStarted = true
                        self.assetWriter.startWriting()
                        self.assetWriter.startSession(atSourceTime: CMSampleBufferGetPresentationTimeStamp(sample))
                    }

Making sure this does not get hit more than once seems to have stabilized things. I am no longer getting recordings I can not play back. Could one of you try ?

mattslight commented 4 years ago

@code-matt which commit / branch is this fix on?

I am having similar issues on playback but running into

{"error":{"domain":"AVFoundationErrorDomain","code":-11829},"target":5175}

I've also tried to playback via setting a timeout after recording completes of 10,000 MS with still same result

mattslight commented 4 years ago

OK I tried testapp in XCode 11.2.1

And same issue out of the box on branch record_inapp_audio_merge

When calling RNRK.previewRecording

2019-11-16 06:21:46.339291+0400 testapp[10032:619955] [Generic] Video /private/var/mobile/Containers/Data/Application/81FD348D-A060-40F6-A4E4-97F1FC510573/Documents/Replays/Recording-1573790972.mp4 cannot be saved to the photo library: Error Domain=AVFoundationErrorDomain Code=-11829 "Cannot Open" UserInfo={NSUnderlyingError=0x281a901b0 {Error Domain=NSOSStatusErrorDomain Code=-12848 "(null)"}, NSLocalizedFailureReason=This media may be damaged., NSURL=file:///private/var/mobile/Containers/Data/Application/81FD348D-A060-40F6-A4E4-97F1FC510573/Documents/Replays/Recording-1573790972.mp4, NSLocalizedDescription=Cannot Open}

code-matt / react-native-replaykit

Recording audio too #10