anarchuser / mic_stream

Flutter plugin to get an infinite audio stream from the microphone
https://pub.dev/packages/mic_stream
GNU General Public License v3.0
100 stars 68 forks source link

Corrupted audio after ByteBuffer change #82

Open DurandA opened 11 months ago

DurandA commented 11 months ago

The commit a3c1ddb42db94ba17aef5ae32f7fb6addec1b12e results in corrupted audio recording on my Android device with PCM-16. From playback, I can recognize the volume but the audio seems fuzzed as if buffer writes started at random places.

I will continue investigating on this issue and document why this change cause issue in the next few days. More context is given in #81 where I misidentified the issue to be with endianness.

DurandA commented 11 months ago

Note to self: try resetting buffer position with data.clear(); before https://github.com/anarchuser/mic_stream/blob/a3c1ddb42db94ba17aef5ae32f7fb6addec1b12e/android/src/main/java/com/code/aaron/micstream/MicStreamPlugin.java#L129-L130

anarchuser commented 11 months ago

Thank you for your investigations. I will give it a look ASAP.

DurandA commented 11 months ago

I gave it another shot. Using data.clear() did nothing which is not surprising:

Note that the value returned by [Buffer.position()](https://developer.android.com/reference/java/nio/Buffer#position()) on this buffer is unchanged after a call to this method. The representation of the data in the buffer will depend on the format specified in the AudioRecord constructor, and will be native endian.

I tried swapping the bytes of the resulting array without success so far. I am not sure what the layout of this direct buffer is. Are you able to playback the audio when it is recorded that way?

anarchuser commented 11 months ago

I cannot, currently, confirm that the audio is actually correct. I need to fix my test setup but am really short on time currently. I do get a waveform with general correspondence to what the mic is recording.

As far as I understood, the ByteBuffer should be the way to go, and I'm getting lost at what the appropriate implementation really is.

anarchuser commented 11 months ago
  1. recorder.read(...) should not modify the position like you cited it. Thus, the current repeat calls to it should be safe. Furthermore, the whole buffer should be completely rewritten every read anyway.
  2. The data should get written as native endian => the simple data.array() should suffice in getting the data in the appropriate byte order.

One way to check for endianness issues is printing some (e.g., first 20) samples straight after recording and then again from the received stream. I'm pretty sure I verified the correctness here at least (I'm still looking for a simple way to automate this testing).

Another potential pitfall may be incorrect application of AudioRecord parameters. I'll need to check that, too.

DurandA commented 11 months ago

Hi @anarchuser and thank you for your time. I didn't have much time to do more research on this yet, for now I am using a fork which reverts a3c1ddb42db94ba17aef5ae32f7fb6addec1b12e.

One way to check for endianness issues is printing some (e.g., first 20) samples straight after recording and then again from the received stream.

This is something I will do for sure.

anarchuser commented 11 months ago

yeah no worries. I'm just maintaining this as a side hobby, I'm happy as long as people find some use in it