mackron / dr_libs

Audio decoding libraries for C/C++, each in a single source file.
Other
1.28k stars 205 forks source link

Advice for internet streaming of FLAC? #236

Closed cosinekitty closed 2 years ago

cosinekitty commented 2 years ago

This is not an issue report as much as seeking advice for a particular use case. I would like to stream FLAC downloaded from http/https.

Streaming has been straightforward for other formats like vorbis and mp3. Both stb_vorbis and minimp3 make it fairly simple to push blocks of data through their respective decoders. Each call to the decoder function returns PCM samples and tells me how many bytes of input it consumed and how many PCM samples it produced. I remove the indicated number of bytes from the front of my buffer, receive more data and append to the end of the buffer, and repeat.

This "push" model is nice because it can be done in a loop inside a single thread: receive a block of data, push through decoder, write PCM. I can throttle playback to the actual rendering rate of my audio hardware, and I can keep going until the stream hits the end or the user decides to pause/stop/seek/skip track. I don't need to guess how much compressed data will produce a given number of PCM samples.

If I understand dr_flac correctly, it provides a "pull" model only, not a "push" model:

/*
Callback for when data needs to be read from the client.
...
Remarks
-------
A return value of less than bytesToRead indicates the end of the stream. Do _not_ return from this callback until either the entire bytesToRead is filled or
you have reached the end of the stream.
*/
typedef size_t (* drflac_read_proc)(void* pUserData, void* pBufferOut, size_t bytesToRead);

It sounds like I can ask drflac_read_pcm_frames_s16 for a given number of PCM frames, but before I do so, I need to either (a) know ahead of time that I already have downloaded enough compressed audio data to satisfy the requested number of frames, or (b) block my thread while I wait for more data to be downloaded.

Is there a recommended way of simulating a "push" approach, where I can update a state machine every time I receive another block of FLAC data, and out pops as many PCM samples as possible? I could create another thread for the FLAC decoder, and provide it a blocking read callback, but that feels complicated and bug-prone, especially when I need to asynchronously abort for a seek or track-skip. I'm hoping there is a more clever way to decode in a single thread, one buffer at a time.

Thanks in advance! I hope my question is expressed clearly.

mackron commented 2 years ago

Yeah this scenario is a bit awkward with dr_flac. There's no push mode so you'll need to make it work with the data callbacks. You'll need to implement that read callback you mentioned and do some data caching yourself. I don't have any particular recommended way of doing it. Basically, if the callback is fired and there's not enough data available to return the entire number of bytes requested by the callback, you'll need to wait for your internal cache to fill up before you return. If you return less than what the callback requests, dr_flac will treat it as though you've reached the end (that's the indicator that dr_flac uses for end-of-file).

cosinekitty commented 2 years ago

That makes sense. After reading through the code some more, I can see that there is no way you can predict how many bytes are required to decode a certain number of frames, because you have to skip over an unpredictable amount of bad-CRC "junk" in order to sync.

I will proceed with creating a dedicated thread for driving dr_flac, and my main push-mode thread will maintain a pair of thread-safe buffers (FLAC in, PCM out) to interact with the dr_flac thread.

Thank you for taking the time to reply and helping me figure out a good approach.