search

jfversluis / Plugin.Maui.Audio

Plugin.Maui.Audio provides the ability to play audio inside a .NET MAUI application
MIT License
264 stars 47 forks source link

Generate the wav file header once recording stopped #115

Closed rezamohamed closed 2 months ago

rezamohamed commented 5 months ago

I am currently storing the audio in my backend storage server as a byte array. The following code works fine for this purpose:

var recordedAudio = await audioRecorder.StopAsync();

using (var stream = new MemoryStream())
                {
                    await recordedAudio.GetAudioStream().CopyToAsync(stream);
                    myByteArray = stream.ToArray();
                }

I can later pull down the byte array from my backend, and then play it back as follows:

                using (var stream = new MemoryStream(myByteArray))
                {
                    var player = AudioManager.Current.CreatePlayer(stream);
                    player.Play();
                }

However, I can't seem to play it in my storage where the raw files are saved.

I was previously using https://github.com/NateRickard/Plugin.AudioRecorder in Xamarin where the files I have stored seemed to play fine. I believe its something to do with the wav header not being written as documented here https://github.com/NateRickard/Plugin.AudioRecorder?tab=readme-ov-file#concurrent-streaming

Is there a way to write said wav header file in this plugin? or am I doing something incorrectly?

To further complicate things, I can't get this method to work on Android for recording/playback.

bijington commented 5 months ago

What platform are you recording the audio on before sending it to your server?

rezamohamed commented 5 months ago

On iOS currently. Also using the newer beta release, tried the older one as well but same thing. Able to record and playback the stream. The stream can’t be played back directly where I am saving on the backend. Files that were recorded by the other plugin mentioned above can be played back directly and also by the currently player.

On Android - the code seems to record. But the playback doesn’t work. Maybe something to do with the header since audio data that was recorded using the other plugin mentioned above can be played back using this player.

Update: Added a zip that contains the file recorded using the other plugin and this one Archive.zip

bijington commented 5 months ago

On iOS we don't do any writing of the header we leave it up to the AVAudioRecorder.

What are you seeing fail when you try to play the files? An exception?

rezamohamed commented 5 months ago

I am realizing what the issue is. If the audio is recorded on an iOS device, it will not play on an Android device with this plugin. This is especially true if you customize the options in iOS. As an example, these options generate a small file size in iOS simulator, but cause an exception when played on the Android simulator.

                var options = new AudioRecordingOptions
                {
                    SampleRate = 16000,
                    Channels = ChannelType.Stereo,
                    BitDepth = BitDepth.Pcm16bit,
                    Encoding = Encoding.Flac,
                    ThrowIfNotSupported = true
                };

In the format saved when I was using https://github.com/NateRickard/Plugin.AudioRecorder it works just fine across any device. There is something about the header I believe.

The Android developer docs however to say that the above format should be fine: https://developer.android.com/media/platform/supported-formats

Here is the exception:

Java.IO.IOException: Prepare failed.: status=0x1 at Java.Interop.JniEnvironment.InstanceMethods.CallVoidMethod(JniObjectReference instance, JniMethodInfo method, JniArgumentValue args) in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/obj/Release/net7.0/JniEnvironment.g.cs:line 20370 at Java.Interop.JniPeerMembers.JniInstanceMethods.InvokeVirtualVoidMethod(String encodedMember, IJavaPeerable self, JniArgumentValue parameters) in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniInstanceMethodsInvoke.cs:line 66 at Android.Media.MediaPlayer.Prepare() in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/obj/Release/net8.0/android-34/mcw/Android.Media.MediaPlayer.cs:line 3857 at Plugin.Maui.Audio.AudioPlayer.PrepareAudioSource() at Plugin.Maui.Audio.AudioPlayer..ctor(Stream audioStream, AudioPlayerOptions audioPlayerOptions) at Plugin.Maui.Audio.AudioManager.CreatePlayer(Stream audioStream, AudioPlayerOptions options) at MyAppName.ViewModels.MyVM.MyVMPageViewModel.PlayVoiceSound(FileData record) in /Users/MyUserName/Development/Repos/MyAppName/MyAppName/ViewModels/MyVM/MyVMPageViewModel.cs:line 116 at CommunityToolkit.Mvvm.Input.RelayCommand`1[[MyAppName.Models.FileData, MyAppName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].Execute(Object parameter) in //src/CommunityToolkit.Mvvm/Input/RelayCommand{T}.cs:line 115 at DevExpress.Maui.CollectionView.DXCollectionView.ItemTap(Int32 visibleIndex) at DevExpress.Maui.CollectionView.DXCollectionView.DevExpress.Maui.CollectionView.Internal.IDXCollectionView.ItemTap(Int32 visibleIndex) at DevExpress.Maui.CollectionView.Android.Internal.ListViewDelegate.DevExpress.Android.CollectionView.IListViewListener.ItemTap(Int32 visibleIndex) at DevExpress.Android.CollectionView.IListViewListenerInvoker.n_ItemTap_I(IntPtr jnienv, IntPtr native__this, Int32 p0) at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PPI_V(_JniMarshal_PPI_V callback, IntPtr jnienv, IntPtr klazz, Int32 p0) in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/Android.Runtime/JNINativeWrapper.g.cs:line 55 --- End of managed Java.IO.IOException stack trace --- java.io.IOException: Prepare failed.: status=0x1 at android.media.MediaPlayer._prepare(Native Method) at android.media.MediaPlayer.prepare(MediaPlayer.java:1337) at crc64a3de810b23927aec.ListViewDelegate.n_itemTap(Native Method) at crc64a3de810b23927aec.ListViewDelegate.itemTap(ListViewDelegate.java:116) at com.devexpress.dxlistview.DXVirtualScrollView.raiseItemTap(DXVirtualScrollView.kt:310) at com.devexpress.dxlistview.core.DragDropManager.onSingleTapUp(DragDropManager.kt:134) at com.devexpress.dxlistview.core.GestureListener.onSingleTapUp(GestureListener.java:119) at android.view.GestureDetector.onTouchEvent(GestureDetector.java:754) at com.devexpress.dxlistview.core.GestureListener.onTouchEvent(GestureListener.java:48) at com.devexpress.dxlistview.DXVirtualScrollView.processTouchEvent(DXVirtualScrollView.kt:305) at com.devexpress.dxlistview.VirtualScrollPanel.dispatchTouchEvent(VirtualScrollPanel.kt:58) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at com.devexpress.dxlistview.DXListViewNative.dispatchTouchEvent(DXListViewNative.kt:339) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at com.android.internal.policy.DecorView.superDispatchTouchEvent(DecorView.java:490) at com.android.internal.policy.PhoneWindow.superDispatchTouchEvent(PhoneWindow.java:1904) at android.app.Activity.dispatchTouchEvent(Activity.java:4377) at crc6488302ad6e9e4df1a.MauiAppCompatActivity.n_dispatchTouchEvent(Native Method) at crc6488302ad6e9e4df1a.MauiAppCompatActivity.dispatchTouchEvent(MauiAppCompatActivity.java:57) at androidx.appcompat.view.WindowCallbackWrapper.dispatchTouchEvent(WindowCallbackWrapper.java:70) at com.android.internal.policy.DecorView.dispatchTouchEvent(DecorView.java:448) at android.view.View.dispatchPointerEvent(View.java:15919) at android.view.ViewRootImpl$ViewPostImeInputStage.processPointerEvent(ViewRootImpl.java:7021) at android.view.ViewRootImpl$ViewPostImeInputStage.onProcess(ViewRootImpl.java:6815) at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:6229) at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:6286) at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:6252) at android.view.ViewRootImpl$AsyncInputStage.forward(ViewRootImpl.java:6417) at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:6260) at android.view.ViewRootImpl$AsyncInputStage.apply(ViewRootImpl.java:6474) at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:6233) at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:6286) at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:6252) at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:6260) at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:6233) at android.view.ViewRootImpl.deliverInputEvent(ViewRootImpl.java:9211) at android.view.ViewRootImpl.doProcessInputEvents(ViewRootImpl.java:9162) at android.view.ViewRootImpl.enqueueInputEvent(ViewRootImpl.java:9131) at android.view.ViewRootImpl$WindowInputEventReceiver.onInputEvent(ViewRootImpl.java:9337) at android.view.InputEventReceiver.dispatchInputEvent(InputEventReceiver.java:267) at android.os.MessageQueue.nativePollOnce(Native Method) at android.os.MessageQueue.next(MessageQueue.java:335) at android.os.Looper.loopOnce(Looper.java:162) at android.os.Looper.loop(Looper.java:294) at android.app.ActivityThread.main(ActivityThread.java:8177) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:552) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:971)

--- End of managed Java.IO.IOException stack trace --- java.io.IOException: Prepare failed.: status=0x1 at android.media.MediaPlayer._prepare(Native Method) at android.media.MediaPlayer.prepare(MediaPlayer.java:1337) at crc64a3de810b23927aec.ListViewDelegate.n_itemTap(Native Method) at crc64a3de810b23927aec.ListViewDelegate.itemTap(ListViewDelegate.java:116) at com.devexpress.dxlistview.DXVirtualScrollView.raiseItemTap(DXVirtualScrollView.kt:310) at com.devexpress.dxlistview.core.DragDropManager.onSingleTapUp(DragDropManager.kt:134) at com.devexpress.dxlistview.core.GestureListener.onSingleTapUp(GestureListener.java:119) at android.view.GestureDetector.onTouchEvent(GestureDetector.java:754) at com.devexpress.dxlistview.core.GestureListener.onTouchEvent(GestureListener.java:48) at com.devexpress.dxlistview.DXVirtualScrollView.processTouchEvent(DXVirtualScrollView.kt:305) at com.devexpress.dxlistview.VirtualScrollPanel.dispatchTouchEvent(VirtualScrollPanel.kt:58) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at com.devexpress.dxlistview.DXListViewNative.dispatchTouchEvent(DXListViewNative.kt:339) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at android.view.ViewGroup.dispatchTransformedTouchEvent(ViewGroup.java:3120) at android.view.ViewGroup.dispatchTouchEvent(ViewGroup.java:2801) at com.android.internal.policy.DecorView.superDispatchTouchEvent(DecorView.java:490) at com.android.internal.policy.PhoneWindow.superDispatchTouchEvent(PhoneWindow.java:1904) at android.app.Activity.dispatchTouchEvent(Activity.java:4377) at crc6488302ad6e9e4df1a.MauiAppCompatActivity.n_dispatchTouchEvent(Native Method) at crc6488302ad6e9e4df1a.MauiAppCompatActivity.dispatchTouchEvent(MauiAppCompatActivity.java:57) at androidx.appcompat.view.WindowCallbackWrapper.dispatchTouchEvent(WindowCallbackWrapper.java:70) at com.android.internal.policy.DecorView.dispatchTouchEvent(DecorView.java:448) at android.view.View.dispatchPointerEvent(View.java:15919) at android.view.ViewRootImpl$ViewPostImeInputStage.processPointerEvent(ViewRootImpl.java:7021) at android.view.ViewRootImpl$ViewPostImeInputStage.onProcess(ViewRootImpl.java:6815) at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:6229) at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:6286) at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:6252) at android.view.ViewRootImpl$AsyncInputStage.forward(ViewRootImpl.java:6417) at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:6260) at android.view.ViewRootImpl$AsyncInputStage.apply(ViewRootImpl.java:6474) at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:6233) at android.view.ViewRootImpl$InputStage.onDeliverToNext(ViewRootImpl.java:6286) at android.view.ViewRootImpl$InputStage.forward(ViewRootImpl.java:6252) at android.view.ViewRootImpl$InputStage.apply(ViewRootImpl.java:6260) at android.view.ViewRootImpl$InputStage.deliver(ViewRootImpl.java:6233) at android.view.ViewRootImpl.deliverInputEvent(ViewRootImpl.java:9211) at android.view.ViewRootImpl.doProcessInputEvents(ViewRootImpl.java:9162) at android.view.ViewRootImpl.enqueueInputEvent(ViewRootImpl.java:9131) at android.view.ViewRootImpl$WindowInputEventReceiver.onInputEvent(ViewRootImpl.java:9337) at android.view.InputEventReceiver.dispatchInputEvent(InputEventReceiver.java:267) at android.os.MessageQueue.nativePollOnce(Native Method) at android.os.MessageQueue.next(MessageQueue.java:335) at android.os.Looper.loopOnce(Looper.java:162) at android.os.Looper.loop(Looper.java:294) at android.app.ActivityThread.main(ActivityThread.java:8177) at java.lang.reflect.Method.invoke(Native Method) at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:552) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:971)

further, I am seeing messages like this in Debug output when playing in iOS, so it may be that the header doesn't get written:

[AudioMetadata] AudioQueueObject.cpp:5942 DetermineMetadataTimestamp unable to obtain valid timestamp, estimate:54459.000000

rezamohamed commented 5 months ago

In addition to the above, I have noticed that the recording on Android specifically has issues: (1) the voice is recorded at a really low volume (3) there is a crackling sound (2) there is a repeat of the last few seconds of the recording always

I created a sample repo to test this (this commit is with this plugin) https://github.com/rezamohamed/MauiAudioTest/tree/fcdba6aafb663a371f73098f55e7e39edad567a2

The Android documentation says that FLAC encoding does work. https://developer.android.com/media/platform/supported-formats

however, when encoding with FLAC, there is an exception thrown:

System.NotSupportedException: Encoding type not supported at Plugin.Maui.Audio.AudioRecorder.SharedEncodingToAndroidEncoding(Encoding type, BitDepth bitDepth, Boolean throwIfNotSupported) at Plugin.Maui.Audio.AudioRecorder.StartAsync(String filePath, AudioRecordingOptions options) at Plugin.Maui.Audio.AudioRecorder.StartAsync(AudioRecordingOptions options) at AudioTest.MainPage.RecordBtn_OnClicked(Object sender, EventArgs e) in /Users/rezamohamed/Development/Projects/AudioTest/AudioTest/MainPage.xaml.cs:line 54 at System.Threading.Tasks.Task.<>c.b128_0(Object state) at Android.App.SyncContext.<>c__DisplayClass2_0.b0() in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/Android.App/SyncContext.cs:line 36 at Java.Lang.Thread.RunnableImplementor.Run() in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/Java.Lang/Thread.cs:line 36 at Java.Lang.IRunnableInvoker.n_Run(IntPtr jnienv, IntPtr native__this) in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/obj/Release/net8.0/android-34/mcw/Java.Lang.IRunnable.cs:line 84 at Android.Runtime.JNINativeWrapper.Wrap_JniMarshal_PP_V(_JniMarshal_PP_V callback, IntPtr jnienv, IntPtr klazz) in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/Android.Runtime/JNINativeWrapper.g.cs:line 22

UPDATE: this plugin to record audio actually works fine on Android and iOS and doesnt have the issues outlined here. https://github.com/NateRickard/Plugin.AudioRecorder The only issue is that the encoding is hard coded so the data is of a bigger size.

This is the commit after switching to this older plugin https://github.com/rezamohamed/MauiAudioTest/commit/64ec94e611535581b3698543cae463692567b728

pierre-galaup commented 4 months ago

Hello,

I too have problems with the repeat of the last few seconds of the recording and the crackling sound at the beginning. As far as I'm concerned, this doesn't happen every time, but very often, especially on very short recordings of 1/2 second or less.

Is there a solution?

jonmdev commented 3 months ago

In addition to the above, I have noticed that the recording on Android specifically has issues: (1) the voice is recorded at a really low volume (3) there is a crackling sound (2) there is a repeat of the last few seconds of the recording always

Regarding the repeat of the last few seconds of hte recording in Android, yes, I am interested to see you observe this. I have seen the same. You can see my bug report here:

https://github.com/jfversluis/Plugin.Maui.Audio/issues/121

I also have in that bug report my method for saving the WAV to disk which plays fine in Windows after at least from Android. I have not yet tested saving to disk from iOS.

borrmann commented 3 months ago

@rezamohamed I am curious why you are unable to play audio that was recorded on a different platform. I have had issues with it initially but then realised I wasnt uploading my files correctly. Maybe that is also whats happening in your case? I forgot to add the correct http header initially, and this is how its working for me:

  1. Record audio. Ensure you are using a format that is supported by all platforms you need, which should be the case for the default values of this library.
  2. Take the created file. I forked the library and added an interface method to retrieve the written file, but I think @jfversluis pointed out that you could also cast the created object to access the class object and not the interface, so you can access the written file directly, without the need to create your own from the audio stream.
  3. upload the file as an IFormFile
  4. ensure that the correct http header is set when uploading the file (e.g. see below)
            var fileType = filePath.EndsWith("mp3") ? "audio/mpeg" : "audio/wav";
            var stream = File.OpenRead(filePath);
            var streamContent = new StreamContent(stream);
            streamContent.Headers.ContentType = MediaTypeHeaderValue.Parse(fileType);

            var multipartContent = new MultipartFormDataContent
                {
                    { streamContent, uriFileParameterName, fileName }
                };

            var response = await _httpClient.PostAsync(uripath, multipartContent, cancellationToken.Value);

As for FLAC support in android, you could alter the method SharedEncodingToAndroidEncoding and add FLAC support. However, currently the plugin only supports writing WAV files, so you would need to take the audio bytes and write your own file. That is the reason I only added the WAV formats when introducing the recording options. You can find currently supported encodings per platform here . Also I am not sure if androids audio recorder supports recording compressed file formats, but if the old library did that, I guess Android may compress the audio internally.

As for the glitching I would also suspect there is maybe something going wrong when writing the file. Since it does not always happen, maybe StopAsync is called multiple times, which could lead to some weird behaviour or bytes being overridden during the writing process? Maybe you could check that by introducing a lock variable for your method? Otherwise you can also find the code that writes the WAV header in aboves link. Maybe there is something wrong?

I dont have any issues with low volume or crackling sound. Are you up and downloading the file before playing it? Maybe this could be fixed by uploading the file in a different way as well?

rezamohamed commented 2 months ago

Changed a few things and got most of it working:

(1) regarding the repeat of the last few seconds, I found that the exraneous Seek and Prepare in the AudioPlayer.Android.cs file is what was causing this. Once I commented these two lines out the playback was fine. Not sure what the intitial intent of these lines were.


void OnPlaybackEnded(object? sender, EventArgs e)
    {
        PlaybackEnded?.Invoke(this, e);

        isPlaying = player.IsPlaying;

        //this improves stability on older devices but has minor performance impact
        // We need to check whether the player is null or not as the user might have dipsosed it in an event handler to PlaybackEnded above.
        if (!OperatingSystem.IsAndroidVersionAtLeast(23))
        {
            //player.SeekTo(0);
            player.Stop();
            //player.Prepare();
        }
    }

(2) The current Android recorder is using AudioRecord, I found this limiting for filetypes and having the need to create our own headers, etc. I replaced AudioRecord with MediaRecorder and this allowed for smaller filetypes, no crackling noise, etc. Updated the StartAsync method with:


if (Android.OS.Build.VERSION.SdkInt >= Android.OS.BuildVersionCodes.S)
            audioRecord = new MediaRecorder(context);
        else
            audioRecord = new MediaRecorder();
        audioRecord.Reset();
        audioRecord.SetAudioSource(AudioSource.Mic);
        audioRecord.SetOutputFormat(OutputFormat.AacAdts);
        audioRecord.SetAudioEncoder(AudioEncoder.Aac);
        audioRecord.SetOutputFile(audioFilePath);
        audioRecord.Prepare();
        audioRecord.Start();

(3) As far as playing files recorded in iOS on an Android, I still havent found the issue. I tried the FLAC type but it still Exceptions out. Currently investigating that. Searches online show many other folks with similar codec issues - https://stackoverflow.com/questions/14645735/can-not-play-recorded-audio-file-from-android-in-ios-5

rezamohamed commented 2 months ago

@borrmann were you able to record on iOS and successfully play on Android? I have tried most of the compatible filetypes but nothing seems to work cross device. I am currently saving as AAC in both device, using OutputFormat.AacAdts and AudioEncoder.Aac in Android (as previous reply) and MPEG4AAC in iOS. Both are on 1 mono channel at 16,000 sample rate.

borrmann commented 2 months ago

@rezamohamed yes, it's working for me to record on iOS and play on Android. I would suspect the issue could be related to the 16,000 sample rate. According to the Android docs 44,100 is the only sample rate that is guaranteed to work on all Android devices. That is for AudioRecord and I guess it should not apply to playing audios, but I suppose it makes sense to always record in the same format. Other than that I use 16 BitsPerSample and Mono as well. I store the files as WAV files.

After that, initially my files weren't playing either, but once I changed the way I uploaded my files and stored them on the server I had no issues anymore. I also play my files using the twilio API in phone calls, and they can be played there as well.

rezamohamed commented 2 months ago

Kinda crazy, but the issue was simply around the extension of the file that the byte[] was getting saved into. I was using a temp file before, but switching it out to a .aac extension solved the problem. 

private static string GetTempFilePath()
{
    //return Path.GetTempFileName();
    return Path.GetTempPath() + Guid.NewGuid() + ".aac";
}
jonmdev commented 2 months ago

@rezamohamed how were you able to get any compressed audio working on Android? It looks to me like AudioRecord is either broken by design (obsolescence) or broken on .NET.

If I run the following code, the only supported codecs that return are:

[0:] SUPPORTS ENCODER: Pcm16bit
[0:] SUPPORTS ENCODER: Pcm8bit
[0:] SUPPORTS ENCODER: PcmFloat
[0:] SUPPORTS ENCODER: Iec61937
[0:] SUPPORTS ENCODER: Pcm24bitPacked
[0:] SUPPORTS ENCODER: Pcm32bit

Here is my test code:

//=======================
//TEST SUPPORT
List<int> supportedEncoders = new();

int numEncodingOptions = Enum.GetNames(typeof(Android.Media.Encoding)).Length; 
int numChannelOptions = Enum.GetNames(typeof(ChannelIn)).Length;

for (int i=0; i < numEncodingOptions; i++) {
    for (int j=0; j < numChannelOptions; j++) {

        string encodingType = ((Android.Media.Encoding)i).ToString();
        int testBufferSize = AudioRecord.GetMinBufferSize(48000, (ChannelIn)j, (Android.Media.Encoding)i);

        Debug.WriteLine("TEST SUPPORT: " + encodingType + " buffer size " + testBufferSize);
        if (testBufferSize > 0) {
            supportedEncoders.Add(i);
        }
    }

}
for (int i=0; i< supportedEncoders.Count(); i++) {
    Debug.WriteLine("SUPPORTS ENCODER: " + (Android.Media.Encoding)(supportedEncoders[i]));
}
//=======================

From your research do you think this is a .NET failure or is it Android design that we are not supposed to be using AudioRecord anymore but rather MediaRecorder?

If AudioRecord is broken/obsolete by design (by .NET or Android) it should be replaced with MediaRecorder inside the Plugin.Maui.Audio formally.

What do you think?

I am trying to sub in MediaRecorder now for AudioRecord but note there is no equivalent isRecording option to determine whether it is in use or not. How did you solve this? Eg. https://stackoverflow.com/questions/13175876/check-to-see-if-android-mediarecorder-is-recording

@bijington & @jfversluis fyi as well.

rezamohamed commented 2 months ago

I had to go with my own implementation for audio record and play back. I used Android MediaRecorder which gives a lot more flexibility and ease of use (eg you don’t need to buffer data and write your own headers). https://learn.microsoft.com/en-us/dotnet/api/android.media.mediarecorder?view=net-android-34.0

jonmdev commented 2 months ago

I had to go with my own implementation for audio record and play back. I used Android MediaRecorder which gives a lot more flexibility and ease of use (eg you don’t need to buffer data and write your own headers). https://learn.microsoft.com/en-us/dotnet/api/android.media.mediarecorder?view=net-android-34.0

Okay, thanks. Yes this appears to be necessary.

Would you be willing to share your code for how you updated AudioRecorder.Android.cs for this? There appears to be no built in way to determine if the MediaRecorder is actively recording as per: https://stackoverflow.com/questions/3458641/how-to-know-whether-a-mediarecorder-is-in-running-state-or-not

So it looks like we need to track this ourselves ... Perhaps just a bool to set on recording start stop in there.

If you are willing to share your code perhaps @bijington & @jfversluis will be willing to update the project for everyone.

For my purposes I have simply copied and pasted the project code (cs files) into my ongoing work project so I can manipulate them directly. But either way it appears AudioRecord should be purged from the project for everyone's benefit.

Thanks again for any thoughts.

bijington commented 2 months ago

If there are improvements or fixes then I'd love to incorporate them 👍

In fact I believe I was reading up on MediaRecorder to be able to provide more support for things like playing from streams, etc.

jonmdev commented 2 months ago

If there are improvements or fixes then I'd love to incorporate them 👍

In fact I believe I was reading up on MediaRecorder to be able to provide more support for things like playing from streams, etc.

Okay so that wasn't too bad. If you just copy and paste this into AndroidRecorder.Android.cs it works (proof of concept):

using System.Diagnostics;
using Android.Content;
using Android.Media;
using Java.IO;

namespace Plugin.Maui.Audio;

partial class AudioRecorder : IAudioRecorder {

    public bool CanRecordAudio { get; private set; }
    private bool isRecordingState = false;
    public bool IsRecording => isRecordingState;

    MediaRecorder? audioRecord; //replaced with MediaRecorder //https://github.com/jfversluis/Plugin.Maui.Audio/issues/115
    string? rawFilePath;
    string? audioFilePath;

    int bufferSize;
    int sampleRate;
    readonly AudioRecorderOptions options;
    int channels;
    int bitDepth;

    public AudioRecorder(AudioRecorderOptions options) {

        var packageManager = Android.App.Application.Context.PackageManager;

        CanRecordAudio = packageManager?.HasSystemFeature(Android.Content.PM.PackageManager.FeatureMicrophone) ?? false;
        this.options = options;
    }

    public Task StartAsync(AudioRecordingOptions options) => StartAsync(GetTempFilePath(), options);
    public Task StartAsync() => StartAsync(GetTempFilePath(), DefaultAudioRecordingOptions.DefaultOptions);
    public Task StartAsync(string filePath) => StartAsync(filePath, DefaultAudioRecordingOptions.DefaultOptions);

    public Task StartAsync(string filePath, AudioRecordingOptions options)  {

        Debug.WriteLine("AUDIO RECORDER START ASYNC STARTED");
        if (CanRecordAudio == false || isRecordingState == true) {

            return Task.CompletedTask;
        }
        options ??= DefaultAudioRecordingOptions.DefaultOptions;

        Debug.WriteLine("AUDIO RECORDER PASSED CAN RECORD AUDIO");

        audioFilePath = filePath;

        var audioManager = Android.App.Application.Context.GetSystemService(Context.AudioService) as Android.Media.AudioManager;

        Debug.WriteLine("AUDIO RECORDER GOT AUDIO MANAGER");

        //set encoding
        Android.Media.AudioEncoder audioEncoder = Android.Media.AudioEncoder.Default;
        Android.Media.OutputFormat outputFormat = OutputFormat.Default;

        if (options.Encoding == Encoding.Aac) {
            audioEncoder = Android.Media.AudioEncoder.Aac;
            outputFormat = OutputFormat.AacAdts;
        }
        else {
            //other formats
        }

        //set channels
        int numChannels = 1;
        if (options.Channels== ChannelType.Stereo) {
            numChannels = 2;
        }

        Debug.WriteLine("AUDIO RECORDER GOT ENCODING");

        //set sample rate
        int sampleRate = options.SampleRate;

        //set bit rate
        int bitRate = options.BitRate;

        Debug.WriteLine("ABOUT TO TRY TO GET ANDROID AUDIO RECORD");

        //create media recorder
        audioRecord = new MediaRecorder(Microsoft.Maui.ApplicationModel.Platform.CurrentActivity.ApplicationContext); //needs context, obsoleted without context//https://stackoverflow.com/questions/73598179/deprecated-mediarecorder-new-mediarecorder#73598440
        audioRecord.Reset();
        audioRecord.SetAudioSource(AudioSource.Mic);
        audioRecord.SetOutputFormat(outputFormat);
        audioRecord.SetAudioEncoder(audioEncoder);
        audioRecord.SetAudioChannels(numChannels);
        audioRecord.SetAudioSamplingRate(sampleRate);
        audioRecord.SetAudioEncodingBitRate(bitRate);
        audioRecord.SetOutputFile(audioFilePath);
        audioRecord.Prepare();
        audioRecord.Start();

        isRecordingState = true;

        Debug.WriteLine("GOT ANDROID AUDIO RECORD");

        return Task.CompletedTask;
    }

    public Task<IAudioSource> StopAsync() {

        Debug.WriteLine("TRY TO STOP ASYNC");
        if (isRecordingState == true) {
            audioRecord?.Stop();
            isRecordingState = false;
        }

        if (audioFilePath is null) {

            throw new InvalidOperationException("'audioFilePath' is null, this really should not happen.");
        }

        //should not be doing header writing or anything here, should not be wav specific
        CopyAudioFile(rawFilePath, audioFilePath);
        Debug.WriteLine("COPY AUDIO FILE FINISHED " + audioFilePath);

        try {
            // lets delete the temp file with the raw data, after we have created the WAVE file
            if (System.IO.File.Exists(rawFilePath))
            {
                System.IO.File.Delete(rawFilePath);
            }
        }
        catch {
            Trace.TraceWarning("delete raw wav file failed.");
        }

        return Task.FromResult(GetRecording());
    }

    IAudioSource GetRecording() {

        if (audioRecord is null || isRecordingState == true || System.IO.File.Exists(audioFilePath) == false) {

            return new EmptyAudioSource();
        }

        return new FileAudioSource(audioFilePath);
    }

    public string? GetAudioFilePath() {
        return audioFilePath;
    }

    static string GetTempFilePath() {

        return Path.Combine("/sdcard/", Path.GetTempFileName());
    }

    void WriteAudioDataToFile() {
        var data = new byte[bufferSize];

        rawFilePath = GetTempFilePath();

        FileOutputStream? outputStream;

        try
        {
            outputStream = new FileOutputStream(rawFilePath);
        }
        catch (Exception ex)
        {
            throw new FileLoadException($"unable to create a new file: {ex.Message}");
        }

        if (audioRecord is not null && outputStream is not null) {

            while (isRecordingState == true) {

                //NOT NEEDED ANYMORE
                //var read = audioRecord.Read(data, 0, bufferSize);
                //outputStream.Write(data, 0, read);
            }

            outputStream.Close();
        }
    }

    void CopyAudioFile(string? sourcePath, string destinationPath)
    {
        long byteRate = sampleRate * bitDepth * channels / 8;

        var data = new byte[bufferSize];

        try
        {
            FileInputStream inputStream = new(sourcePath);
            FileOutputStream outputStream = new(destinationPath);

            if (inputStream?.Channel is not null)
            {
                var totalAudioLength = inputStream.Channel.Size();
                //var totalDataLength = totalAudioLength + 36;
                var totalDataLength = totalAudioLength;

                //WriteWaveFileHeader(outputStream, totalAudioLength, totalDataLength, sampleRate, channels, byteRate);

                while (inputStream.Read(data) != -1)
                {
                    outputStream.Write(data);
                }

                inputStream.Close();
                outputStream.Close();
            }
        }
        catch (Exception ex)
        {
            // Trace the exception
            Trace.WriteLine($"An error occurred while copying the wave file: {ex.Message}");
            Trace.WriteLine($"Stack Trace: {ex.StackTrace}");
        }
    }

}

You can expand AudioRecorderOptions to have bitrate also as an int which I will be adding on my end. You can expand the Encoding enum to provide the extra options like AAC.

Seems not too bad. Only quirk I see is I am not sure how to make MediaRecorder create plain wav's. If you select an encoder the only options are compressed ones. But 99% of people will want audio compression anyway. Not sure if I'm missing something. Doesn't matter to me as I need compressed audio.

I have copied and pasted the cs files into my project so I won't be pushing any updates to Plugin.Maui.Audio but I share this if you want to update it.

I think the project should support audio compression as no likely user wants to record wav files on mobile Maui apps. This solves the Android component in any case if you're interested.

I will have to figure out iOS next.

rezamohamed commented 2 months ago

If there are improvements or fixes then I'd love to incorporate them 👍

In fact I believe I was reading up on MediaRecorder to be able to provide more support for things like playing from streams, etc.

Yep, most of the issues that I had on this thread went away when I switched over to mediarecorder.

SchittkowskiMS commented 1 month ago

For anyone having issue with playing audio files on android that are recorded on ios. Currently i got it working with the help of @rezamohamed post here

Kinda crazy, but the issue was simply around the extension of the file that the byte[] was getting saved into. I was using a temp file before, but switching it out to a .aac extension solved the problem.

private static string GetTempFilePath()
{
    //return Path.GetTempFileName();
    return Path.GetTempPath() + Guid.NewGuid() + ".aac";
}

When feeding this string into the AudioRecorder.StartAsync function i got an exception, but changing ".aac" to ".wav" solved it. I am now able to record on ios, upload the file to a server, download it on an android device and starting the replay. On android i didnt change anything, just calling the AudioRecorder.StartAsync function without any parameters.

Still gonna do some further testing but this solved it for now

rezamohamed commented 1 month ago

@SchittkowskiMS I'm not sure what your use case is, but recording to WAV was creating really large files for me. Once I switched over to MediaRecorder, I was able to compress and save as AAC, and the file size almost dropped 90%. For my use case, I was saving this onto cloud storage, so the space savings were worth it. Also, the transmission time reduced since it was smaller.

SchittkowskiMS commented 1 month ago

Yes would be nicer to have smaller filesize but for my current application i think this will do until maybe someone (@jonmdev ?) updates this package

bijington commented 1 month ago

It sounds like the changes would benefit the plugin and others so if anyone is willing to submit a PR we would gladly include it