search

sbooth / SFBAudioEngine

A powerhouse of audio functionality for macOS and iOS
https://sbooth.github.io/SFBAudioEngine/
MIT License
552 stars 87 forks source link

Unable to stream MP3/WAV and possibly other formats #258

Closed lyarenei closed 4 months ago

lyarenei commented 1 year ago

Hi, I'm currently evaluating SFBAudioEngine for my future iOS/macOS app using the SimplePlayer demo app. But when I try to play this mp3 sample remotely, it refuses to play, saying it's not a recognized format.

The logs are not saying much, just TagLib reports that the file could not be opened and the download gets cancelled. It also looks like that 2 sessions for the download are spawned for some reason?

TagLib: Could not open file /mp3/sample-15s.mp3
2023-04-17 19:21:57.966166+0200 SimplePlayer[26813:14498664] [InputSource] NSURLSessionData error: Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=https://download.samplelib.com/mp3/sample-15s.mp3, NSErrorFailingURLKey=https://download.samplelib.com/mp3/sample-15s.mp3, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <F4F92142-6FF9-48AE-96CB-ADE6A29A6A26>.<1>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <F4F92142-6FF9-48AE-96CB-ADE6A29A6A26>.<1>, NSLocalizedDescription=cancelled}
2023-04-17 19:21:57.966174+0200 SimplePlayer[26813:14498624] [InputSource] NSURLSessionData error: Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=https://download.samplelib.com/mp3/sample-15s.mp3, NSErrorFailingURLKey=https://download.samplelib.com/mp3/sample-15s.mp3, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <ED3E8BA8-698D-4755-AE5C-5EA03DB53DB3>.<1>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <ED3E8BA8-698D-4755-AE5C-5EA03DB53DB3>.<1>, NSLocalizedDescription=cancelled}
2023-04-17 19:21:57.969161+0200 SimplePlayer[26813:14498624] [InputSource] HTTP status: 200 no error

If I try a wav sample from the same site, SimplePlayer reports it as unsupported:

TagLib: Could not open file /wav/sample-12s.wav
2023-04-17 19:24:24.044094+0200 SimplePlayer[26813:14497021] [AudioDecoder] sf_open_virtual failed: Format not recognised.
2023-04-17 19:24:24.911377+0200 SimplePlayer[26813:14499447] [InputSource] HTTP status: 200 no error

I've tried multiple samples from multiple servers, but the errors are same. Local playback of these two samples mentioned above works without any issues. The only format I was able to stream successfully was FLAC.

I also think this might be related, or maybe it's even discussing the same issue: https://github.com/sbooth/SFBAudioEngine/discussions/206, but that looks like it ended up unresolved.

Unfortunately I don't know anything about objc, othwerwise I'd try to debug this further. Can you please take a look into it? Let me know if you need more details. Thanks.

sbooth commented 1 year ago

I think this is the same issue discussed in #206 and I think it boils down to treating -openReturningError: synchronously when it is inherently asynchronous in this case due to network requests.

This problem occurs because there are attempts to read data from the input source (via sf_open_virtual in the case of WAVE, but it will differ by format) before the NSURLSessionDataTask has made any calls to the delegate returning data from the request. When this happens the input source returns a zero length which is typically reported as an unsupported format.

I feel like this is a design flaw in the way that HTTP input sources are handled. Years ago HTTP input was implemented using CFReadStream (see https://github.com/sbooth/SFBAudioEngine/blob/legacy/Input/HTTPInputSource.cpp if you're curious) which seemed to work at the time with the library's architecture. In the transition to Objective-C and Swift CFReadStream was replaced with the current design.

I don't have a great fix off the top of my head, but one possibility is exposing SFBHTTPInputSource as a public class and requiring users to ensure the network has returned data before trying to create a decoder from the input source. It's also possible that the library could handle that internally although I'm not sure what that would entail.

In a perfect world perhaps all reads would be asynchronous (taking a completion handler in Objective-C and using async in Swift) but that would be significant change to the way things work.

Bringing in @NattyNarwhal for any suggestions.

NattyNarwhal commented 1 year ago

I moved my application to AVFoundation, which from what I observe (could be inaccurate!) seems block only to load just enough of the file (metadata, maybe initial audio frames), then continues to load the rest in asynchronously behind the scenes; file progress is observable. You can also set an option to block a bit longer to load more data for more accurate seeks if the container can't be accurate about it.

In the SFBHTTPInputSource case, it seemed all the libraries it delegates to would fail because nothing was loaded of the file yet.

sbooth commented 5 months ago

For now I've removed SFBHTTPInputSource.