Provide Raw Audio Data Capture from AudioPlayers

[Sungray]

Is your feature request related to a problem? Please describe. I need to capture raw audio data from AudioPlayer instances to send the app's audio to external services such as Discord or a WebSocket server.

Describe the solution you'd like Implement a mechanism to allow users to register callbacks to AudioPlayer instances that provide access to the audio data buffers being processed. These callbacks should be invoked with the raw PCM data at the same point where the audio is sent to the output. The library should expose the audio data in a standard PCM format, such as PCM 16-bit or PCM 32-bit float, so that it can be processed or streamed as needed.

Describe alternatives you've considered I implemented the Audio Playback Capture API to capture the app's audio and process the PCM stream correctly. However, any effects applied to the AudioPlayer instances, such as volume changes, are ignored. This is problematic when managing multiple AudioPlayer instances with different volume levels and effects. Synchronizing volumes natively is not feasible in this scenario.

Additional context No additional context.

[ryanheise]

Although I'm not personally motivated by this feature, can you expand on your use case? I am unclear on why volume is part of the use case. If the phone user controls the volume data with the physical buttons, this will not show up in the PCM data that comes out of the audio player as it's handled by the OS itself. At best, you could capture the volume changes when the player's setVolume is used instead. Capturing the output of audio effects might present a complication.

As a starting point for implementation, the visualizer provides a low resolution version of the audio data for visualisation purposes. The iOS implementation gives an idea of how this could be captured at full resolution, while the Android implementation would need to be done very differently, and again, the audio effects might present a complication. One other complication is the efficiency of sending such high resolution data over method channels.

In any case, I will leave this as a feature for someone else to contribute if they are interested in contributing.

[Sungray]

Well, for example, the user can decide to switch from one playing audio category to another one. In such case, the first category performs a fade out, and the second a fade in, at the same time. The fade effect is entirely lost with the Audio Playback Capture API because the volume is ignored. I do not know the exact reason, but it's probably that the software effects applied by just_audio are not part of the raw data captured by the Audio Playback Capture API.

There could also be a volume slider in the app, which is also ignored by the Audio Playback Capture API.

I was also considering using this data stream to enable a chromecast/airplay implementation. Basically the point is to have the exact audio output mirrored remotely.

[ryanheise]

OK, thanks for clarifying.