MediaRecorder implementation for iOS

[rostopira]

I can barely read Objective-C code, so help wanted)

[aryzhov]

Could you outline the needed functionality and volume of work?

[rostopira]

For basic functionality we need capturing, encoding and muxing of video/audio track Current dart api here: https://github.com/cloudwebrtc/flutter-webrtc/blob/master/lib/media_recorder.dart Android implementation here: https://github.com/cloudwebrtc/flutter-webrtc/pull/46 It's originally based on original implementation of https://webrtc.googlesource.com/src/+/master/sdk/android/api/org/webrtc/VideoFileRenderer.java Just added encoding, muxing and recording of local audio (unfortunately impossible to record any other AudioTrack due to current WebRTC implementation) However, I've didn't find any hints for encoding or even writing to file in ios part of that repo

[rostopira]

I've made some progress with it, but currently busy with another project. Will push it to repo on next week

[rostopira]

It's almost there! If somebody wants to test it:

  flutter_webrtc:
    git:
      url: https://github.com/cloudwebrtc/flutter-webrtc
      ref: ios_recorder

UPD: currently only remote audio track is supported

[rostopira]

I were unable to find any solution for recording local audio track in WebRTC pod If you are interested in this feature, star this issue: https://bugs.chromium.org/p/webrtc/issues/detail?id=10166

[aryzhov]

Thank you, Dima, for working on this. I just starred it.

On Mon, Apr 15, 2019 at 4:05 AM Dima Rostopira notifications@github.com wrote:

I were unable to find any solution for recording local audio track in WebRTC pod If you are interested in this feature, star this issue: https://bugs.chromium.org/p/webrtc/issues/detail?id=10166

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cloudwebrtc/flutter-webrtc/issues/47#issuecomment-483207475, or mute the thread https://github.com/notifications/unsubscribe-auth/AAlvEq6D7soMIEISmZN4mHjztB6BslO_ks5vhFz8gaJpZM4ZjtQ5 .

[sam0829]

@rostopira Hi, I am trying to record using the mentioned branch but the app crashes at _audioSource->AddSink(_bridge); inside FlutterRTCAudioSink. The crash isn't revealing any information and just says EXC_BAD_ACCESS code 2. Did you face any such errors or do you know what's going on here?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mrcendre]

Hello @rostopira @cloudwebrtc, we are attempting to record a remote audio track to a file on iOS, using your implementation of FlutterRTCAudioSink. We have experienced the same issue as described by @sam0829 with the every version of the WebRTC-SDK pod. However, when using the GoogleWebRTC pod up to a certain version, the audio sink works fine and we can successfully write the audio data to our file.

Here is a detailed trace for the error, with a debug build of WebRTC :

Fatal error (Thread 1) : EXC_BAD_ACCESS (code=2, address=0x106ebf864)

#0  0x0000000107bf3ce8 in absl::optional_internal::optional_data_dtor_base<bool, true>::optional_data_dtor_base<bool>(absl::in_place_t, bool&&) at webrtc/src/third_party/abseil-cpp/absl/types/internal/optional.h:133
#1  0x0000000107bf3cb8 in absl::optional_internal::optional_data_base<bool>::optional_data_base<bool>(absl::in_place_t, bool&&) at webrtc/src/third_party/abseil-cpp/absl/types/internal/optional.h:141
#2  0x0000000107bf3c84 in absl::optional_internal::optional_data<bool, true>::optional_data<bool>(absl::in_place_t, bool&&) at webrtc/src/third_party/abseil-cpp/absl/types/internal/optional.h:184
#3  0x0000000107bf3c3c in absl::optional<bool>::optional<bool, false>(bool&&) at webrtc/src/third_party/abseil-cpp/absl/types/optional.h:178
#4  0x0000000107bd4fa4 in absl::optional<bool>::optional<bool, false>(bool&&) at webrtc/src/third_party/abseil-cpp/absl/types/optional.h:178
#5  0x00000001081d50e4 in cricket::AudioOptions::AudioOptions() at webrtc/src/api/audio_options.h:39
#6  0x00000001081d51c4 in cricket::AudioOptions::AudioOptions() at webrtc/src/api/audio_options.cc:37
#7  0x000000010820788c in webrtc::AudioSourceInterface::options() const at webrtc/src/api/media_stream_interface.cc:35
#8  0x0000000102412df4 in -[FlutterRTCAudioSink initWithAudioTrack:] at flutter-webrtc/ios/Classes/FlutterRTCAudioSink.mm:24
#9  0x000000010241417c in -[FlutterRTCMediaRecorder initWithVideoTrack:rotationDegrees:audioTrack:outputFile:] at flutter-webrtc/common/darwin/Classes/FlutterRTCMediaRecorder.m:33
#10 0x00000001024256b0 in -[FlutterWebRTCPlugin handleMethodCall:result:] at flutter-webrtc/common/darwin/Classes/FlutterWebRTCPlugin.m:1020
#11 0x0000000109fc67b8 in __45-[FlutterMethodChannel setMethodCallHandler:]_block_invoke ()
#12 0x0000000109affab8 in flutter::PlatformViewIOS::HandlePlatformMessage(std::__1::unique_ptr<flutter::PlatformMessage, std::__1::default_delete<flutter::PlatformMessage> >) ()
#13 0x0000000109e8d95c in std::__1::__function::__func<fml::internal::CopyableLambda<flutter::Shell::OnEngineHandlePlatformMessage(std::__1::unique_ptr<flutter::PlatformMessage, std::__1::default_delete<flutter::PlatformMessage> >)::$_15>, std::__1::allocator<fml::internal::CopyableLambda<flutter::Shell::OnEngineHandlePlatformMessage(std::__1::unique_ptr<flutter::PlatformMessage, std::__1::default_delete<flutter::PlatformMessage> >)::$_15> >, void ()>::operator()() ()
#14 0x0000000109db4f80 in fml::MessageLoopImpl::FlushTasks(fml::FlushType) ()
#15 0x0000000109db8ce4 in fml::MessageLoopDarwin::OnTimerFire(__CFRunLoopTimer*, fml::MessageLoopDarwin*) ()
#16 0x00000001944aaf80 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ ()
#17 0x00000001944aab80 in __CFRunLoopDoTimer ()
#18 0x00000001944aa054 in __CFRunLoopDoTimers ()
#19 0x00000001944a438c in __CFRunLoopRun ()
#20 0x00000001944a36a8 in CFRunLoopRunSpecific ()
#21 0x00000001aab53570 in GSEventRunModal ()
#22 0x0000000196dc1370 in -[UIApplication _run] ()
#23 0x0000000196dc68ec in UIApplicationMain ()
#24 0x000000010230ef90 in main at flutter_app/ios/Runner/Classes/AppDelegate.swift:6

FlutterRTCAudioSink.mm :

...

17    - (instancetype) initWithAudioTrack:(RTCAudioTrack* )audio {
18        self = [super init];
19
20        webrtc::scoped_refptr<webrtc::AudioSourceInterface> audioSourcePtr = audio.source.nativeAudioSource;
21        _audioSource = static_cast<webrtc::AudioSourceInterface *>(audioSourcePtr.get());
22        _bridge = new AudioSinkBridge((void*)CFBridgingRetain(self));
23    
24        _audioSource->AddSink(_bridge);
25    
26        return self;
27    }

...

There is absolutely no information about this specific Abseil error on the web.

After some research, we can successfully call AudioSourceInterface->AddSink when using the GoogleWebRTC pod up to version 1.1.27564 (just like @rostopira did), but the error always occurs in every build starting with version 1.1.27724 and in every version of the WebRTC-SDK pod.

Historically, it is our understanding that GoogleWebRTC has been deprecated in 2020, in favor of the WebRTC-SDK spec repository. However, has anything changed in the WebRTC source since then that could explain this error ? Could this be caused by missing or a bad compilation options that would break the Abseil optionals' internal logic ? Maybe with the way C++ initializes structs with default values ?

Thanks for your help !

[bantunes22]

@rostopira Did you finish this implementation? This issue is closed but when using the MediaRecorder::start(...), I'm getting this:

Unhandled Exception: MissingPluginException(No implementation found for method startRecordToFile on channel FlutterWebRTC.Method)

Is there anything I'm missing? I was digging into the code and on the iOS side I didn't find anything with the "startRecordToFile".

Using the lastest version available on the pub.dev.

On the pub.dev, there is a warning sign on the MediaRecorder, but no additional information about that warnings.

Is there anything I can do to help? I know Obj-C, so if there is no implementation yet and you can guide me to the proper documentation on this, I can try to help on this one.

I went to https://github.com/cloudwebrtc/flutter-webrtc but I was able to find the ios_recorder branch like you mentioned here:

https://github.com/flutter-webrtc/flutter-webrtc/issues/47#issuecomment-480195469

Thanks

[rostopira]

@bantunes22 checkout PR #1189

It's very hacky, so I decided not to merge it. I were going to create patch for WebRTC source itself, however still can't find time for that.

[bantunes22]

@bantunes22 checkout PR #1189

It's very hacky, so I decided not to merge it. I were going to create patch for WebRTC source itself, however still can't find time for that.

Thats awesome. I will have a look into it, and give some feedback. Thanks for the work, and let me know if ther is anything I can help.

[fantasyEthan]

Any idea of recording both video and audio on iOS on my iPhone? I also encountered No implementation found for method startRecordToFile on channel FlutterWebRTC.Methodand No implementation found for method stopRecordToFile on channel FlutterWebRTC.Method when running on iOS. How can I solve it or find a substitute way. Thanks

[patelnirav48]

@rostopira @bantunes22 Could you please help to tell when iOS recording support going to available?