db-dblank
commented
8 months ago I have a similar scenario where I need to stream audio coming from a websocket. We are using ElevenLabs and the audio comes in small chunks from a websocket.
class StreamSource extends StreamAudioSource {
StreamSource(this.stream);
final Stream<List<int>> stream;
@override
Future<StreamAudioResponse> request([int? start, int? end]) async {
return StreamAudioResponse(
sourceLength: null,
contentLength: null,
offset: 0,
stream: stream,
contentType: 'audio/mp3',
);
}
}
final audioPlayer = AudioPlayer();
final streamCtrl = StreamController<List<int>>.broadcast();
final ws = WebSocketChannel.connect(Uri.parse('wss://example.com'));
ws.stream.listen((event) async {
if (event.type == 'start') {
await audioPlayer.setAudioSource(StreamSource(streamCtrl.stream));
await audioPlayer.play();
}
streamCtrl.sink.add(base64Decode(event.audio));
});No audio is being played this way. If a play each chunk separately by setting setAudioSource for each with another StreamAudioSource implementation, it plays but there is a gap between each audio chunk.
@ryanheise is there a way this could be done? Am I doing something wrong? Thanks
ryanheise
artyomkonyaev
fvisticot
Which API doesn't behave as documented, and how does it misbehave? When playing a
StreamAudioSourcethat returns responses supporting range requests, withnulloffset and lengths, and returning WAV PCM data with a header indicating the total length is greater than the returned data, no subsequent range requests are made to the source. Tested in Chrome.Minimal reproduction project https://github.com/yaakovschectman/flutter-just-audio-test
To Reproduce (i.e. user steps, not code) Steps to reproduce the behavior:
flutter run -d chromein the above project.Error messages No error message.
Expected behavior I expect that the player would send an initial request to the
StreamAudioSourceand receive a response that indicates the source a) supports range requests, and b) still has more data to send. This should result in an effectively continuous sound playing when the button is pressed, as the player keeps requesting new data to play. Instead, the sound plays for 1 second (the duration of the initial request's response), and then goes silent. A print debug reveals no subsequent requests are received until the button is pressed again, callingplay()again.Screenshots No applicable screenshots.
Desktop (please complete the following information):
Smartphone (please complete the following information): Not tested on smartphone.
Flutter SDK version
Additional context As an example application, suppose I am developing an application that generates audio in realtime and plays it over the speakers, such a DSP application or synthesizer. In this case, I would need to continuously stream small buffers of fresh audio data from the source to the player. From the documentation, it seems this should be done by subclassing
StreamAudioSourceto return the chunks of data to the requester. I've not been able to figure out how to make this actually work, or if there is a preferred method instead.