LockCachingAudioSource encoding prevents playback from Android to iOS

[sallypeters]

Which API doesn't behave as documented, and how does it misbehave? Name here the specific methods or fields that are not behaving as documented, and explain clearly what is happening.

LockCachingAudioSource

• Doesnt enable playback on iOS devices if Audio was recorded initially on an Android device.

Minimal reproduction project Provide a link here using one of two options:

1. Fork this repository and modify the example to reproduce the bug, then provide a link here.
2. If the unmodified official example already reproduces the bug, just write "The example".

Here is a link to the modified just_audio repo :

https://github.com/sallypeters/just_audio.git

and here is a link to the code I modified :

https://github.com/sallypeters/just_audio/blob/d6d0d83c2bb68ea9eb951727f11d25d1185f3e31/just_audio/example/lib/example_caching.dart

To Reproduce (i.e. user steps, not code) Steps to reproduce the behavior:

Run code from my repo. You need to run the example_caching.dart.

Uncomment the URLS in the code :

(1) - This works on both iPhone & Android (2) - This only works on Android

Error messages

Error loading audio source: (-11800) The operation could not be completed

Expected behavior Should be able to record audio on Android/iPhone and have same audio played on both Android/iPhone

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. MacOS + Big Sur]
• Browser [e.g. Chrome Version 94.0.4606.71 (Official Build) (x86_64)]

Smartphone (please complete the following information):

• Device: [e.g. iPhone 11 Pro Max]
• OS: [e.g. iOS 15]

Flutter SDK version

[✓] Flutter (Channel stable, 2.5.2, on macOS 11.6 20G165 darwin-x64, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.0)
[✓] Xcode - develop for iOS and macOS
[✓] Chrome - develop for the web
[✓] Android Studio (version 4.0)
[✓] IntelliJ IDEA Community Edition (version 2018.2.3)
[✓] VS Code (version 1.60.2)
[✓] Connected device (13 available)

Additional context Add any other context about the problem here.

[sallypeters]

I've spent more time looking at this and can add :

When recorded on Android, saved as an AAC file - uploaded to a server and the link opened on an iOS device

The code below does not play anything even though the cached file is actually created and saved in the supplied location

Uri uri = Uri.parse(widget._tuple.playbackFile);
    var audioSource = LockCachingAudioSource(
      uri,
      cacheFile: File(cacheFile),
    );
    await _soundPlayer.setAudioSource(audioSource, preload: false);

However - if we instead do not use LockCachingAudioSource and use setUrl the same resource URL plays on iOS without issue.

Furthermore - the files created on Android and saved on an iOS device can be dropped onto an Android Simulator and played without issue.

When recorded on iOS, saved as an AAC or MP3 extension file - uploaded to a server and the link opened on an Android device

There is no issue. File is persisted in cache. File is then played. Works on both Android and iOS

Other Relevant Info

If we record in Android - encode as AAC and save with extension as AAC. Then upload to any server so we create a shared URL. Then open this URL on an iOS device via LockCachingAudioSource - as mentioned the cached file gets created (Assuming we create our own cached file) but cannot play. More importantly - setFilePath(local cached File) will not work either but setUrl(our original URL) will (i.e we can open and play an Android recorded audio file AAC on iOS).

[ryanheise]

Thanks, @sallypeters

I plan to look at this today.

[ryanheise]

I've just tried opening both URLs in Safari on mac, and the same thing happens: the first URL works, the second URL doesn't. I would suggest this indicates an issue with the encoding of your file. It is not surprising to see URLs that work on one platform and not on another, since different platforms have different support for different audio codecs, as well as there being other differences in the handling of HTTP headers and file name extensions.

[ryanheise]

This may also indicate that the audio recorder plugin you're using has a bug on the Android side in which it is not encoding the file correctly.

[sallypeters]

I have recorded audio also on an Android S10+, i.e just using the standard voice recorder (No flutter sound recording plugin) that ships with the phone.

I then take the audio file and again add this audio file to a cloud based service. With the URL I attempt to access this via the just_audio caching app and again the same results appear, i.e Android is okay - iOS nothing happens but if we don't use the LockCachingAudioSource, and use setURL - the audio plays, so I think this proves the encoding is sound and the issue is with LockCachingAudioSource.

In order to progress with my needs , I now have code that checks if the client is Android and if so I use the LockCachingAudioSource (There is no issue with this class/API if the client is either Android or iOS) - it works as intended.

If the client is iOS - I now provide caching myself outside of LockCachingAudioSource , I use setUrl to start the initial download and then use HTTP services to download the URL simultaneously as just audio is streaming from the URL. When the URL completes its download, I rename the downloaded file with an extension of m4a (This is important for iOS otherwise setFilePath will fail) and then switch from the URL playback to a file based playback with no apparent detectable change over.

This all works seamlessly. I get caching on the iOS side since I have helper methods that do something like :

if (!DeviceHelper.isAndroid) {
      /// There is an issue with iOS and playing recorded Audio from Android - for this reason we cannot
      /// use LockCachingAudioSource so until the issue is fixed, we need to use our own cache ...

      /// Note - in order for this to work , we must ensure that under iOS we rename the files that we have
      /// in Google Storage to have an extension of .m4a otherwise setFilePath will fail.
      if (await cacheManager.isInCache(audioFile)) {
        _soundPlayer.setFilePath(cacheFile);
        return;
      } else {
        /// Not in Cache - setUrl actually works , we can use that
        _soundPlayer.setUrl(widget._tuple.playbackFile);
        _soundPlayer.play();
... Caching code .....
}else{
/// Standard lockCachingAudioSource code will work for android 

The other thing which may be relevant here is depending on the cloud service/platform one employs for hosting audio/images - generally the encoding is derived from the extension , however , one can also (in the example of google cloud storage) apply added metadata which helps with the encoding - i.e we can do something like this :

 final UploadTask task = storageReference.putData(
      bytes,SettableMetadata(contentType: 'audio/aac'));
    );

This will affect how the bytes are encoded and stored in Storage but in my tests, this actually adversely affects just_audio

So to summarise - if as you say a stream of bytes has been incorrectly encoded then I would expect all AudioSource descendants to fail but as pointed out, in my tests - setUrl works for me.

It would appear that unlike setUrl - lockCachingAudioSource and setFilePath is very much dependent on the correct FILE EXTENSION in order for them to correctly function in iOS.

Furthermore - yes you are right, the 2nd URL will not play if you simply drop it into Safari (Chrome is fine , it works) - but that use case is insufficient to ascertain the validity of the issue that has been raised. With defensive code and the approach I have mentioned above , I can get both URL's working (Using just_audio , just not using LockCachingAudioSource) - especially the 2nd URL. For my own needs, I dont need LockCachingAudioSource to be fixed since I have written a workaround - I was only extending this effort and feedback to enable further research and investigation to get a resolution for potentially the benefit of other users of just_audio, I'm fine if you want to close this as I have my workaround.

[ryanheise]

Chrome doesn't use the Apple decoders, it uses its own cross-platform decoders. Safari on the other hand uses Apple's own platform decoders which are the same I use within this plugin, hence it follows the same restrictions.

One difference in encoding between the two files you supplied is that in the second the mdat box comes before the moov box which is known to cause playback issues over the web, but not for files, since for files random access is easier and the player can jump around more easily. On the web, this is what range requests are designed to support and indeed Apple does require the server to support range requests. Your server does support these, but the "lock caching" algorithm works by first downloading enough of the file linearly before servicing a particular range request and blocking until then, so it is still better to optimise the audio encoding for web playback if you want to use lock caching. I did eventually want to add other caching algorithms but the lock caching algorithm was the easiest one to begin with.

There is a bug in LockCachingAudioSource in that it does not correctly cache the content type header, although after fixing this in my local branch it did not have any effect on your issue.

In any case, I tried playing your second audio URL with setUrl and it failed with exactly the same error, consistent with Safari's behaviour. So I don't think your workaround is reliable and it would probably be better when using one of the Apple platforms to follow Apple's own requirements when playing audio.