Play audio directly from memory?

[hacker1024]

Is your feature request related to a problem? Please describe. The APIs I'm working with use XOR encryption. (For an example, they can be decrypted by this tool.)

Describe the solution you'd like It'd be great if I could pass the decrypted MP4 audio directly to this plugin, or, even better, have the plugin decrypt on-the-fly (with concatenated audio support for gapless playback).

I know it's a big ask, and a niche feature - I'm happy to try and implement this myself, but what do you think about the two methods? I think seamless plugin-side decryption sounds better as it's easier to use the gapless playback features.

Describe alternatives you've considered I suppose I could try writing to a file and playing it, but that would cause unnecessary wear on a device's internal storage.

Additional context Nope.

[ryanheise]

You might consider looking at the proxy server in the code and doing something similar to that outside the plugin. This would give you gapless playback in memory without adding a niche feature directly to the plugin.

[hacker1024]

Right, that's another option. I considered it, but I'm not sure of that would have to use a port that could be used by something else on the device.

[ryanheise]

You can use an ephemeral port as done in _ProxyHttpServer.

[hacker1024]

Thanks, I'll take a look.

Also - while decryption is a fairly uncommon feature, the ability to play audio from a stream of bytes would be (in my opinion) a nice addition to the plugin.

Would you accept a pull request that allows such a thing?

[ryanheise]

Now that could be a really useful feature, I agree. I would be happy to accept a PR.

There's one other feature that should eventually go into the proxy code which is the ability to write the downloaded file to a cache as it's being downloaded, so if you plan to add code to the proxy, it would be good to just keep that other feature in the back of your mind so that it still keeps the door open for that other feature to be easily added later.

[hacker1024]

There's one other feature that should eventually go into the proxy code which is the ability to write the downloaded file to a cache as it's being downloaded, so if you plan to add code to the proxy, it would be good to just keep that other feature in the back of your mind so that it still keeps the door open for that other feature to be easily added later.

Will do, thanks for the heads up.

I'm using your proxy suggestion for now, and it works really well - thanks for that. The one problem is that the server is accessible with a web browser, which isn't ideal (especially as this method is used in this generic plugin) as private data can be exposed.

Do you know of any way to remedy that?

Since I have a working solution at the moment, my PR idea is less of a priority; however, I think it's cleaner (and potentially less resource-intensive) than using a proxy (if, of course, the implementation does not use a proxy itself).

I'll have a go at implementing it eventually, but it might not be for a while.

[hacker1024]

Oh, and in case anyone wants to do something similar, here's my implementation.

/// This class handles decryption of XOR encrypted media.
/// It provides a proxy server that acts as middleware
/// between audio plugins and media servers,
/// decrypting data on-the-fly.
class _XORDecryptionMediaProxy {
  // Content type used in responses.
  static final _contentType = ContentType.parse('audio/mp4');

  // Create a client to make requests with.
  static final _client = Client();

  // The HTTP server.
  HttpServer _server;

  // These maps hold data about CDN hostnames and decryption
  // keys against a token unique to each encrypted track media
  // URL.
  final _keyMap = <String, String>{};
  final _cdnHostMap = <String, String>{};

  // Save a remote URL CDN hostname and decryption key
  // against it's unique token.
  Uri addUrl(String url, String key) {
    final remoteUri = Uri.parse(url);

    final localUri = remoteUri.replace(
      scheme: 'http',
      host: InternetAddress.loopbackIPv4.address,
      port: _server.port,
    );

    final token = remoteUri.queryParameters['token'];

    _cdnHostMap[token] = remoteUri.host;
    _keyMap[token] = key;

    return localUri;
  }

  // Starts the server.
  Future<void> start() async {
    // Start an HTTP server on a port decided by the OS.
    _server = await HttpServer.bind(InternetAddress.loopbackIPv4, 0);

    // Handle requests.
    _server.listen((request) async {
      // Record a token unique to each URI.
      final token = request.uri.queryParameters['token'];

      // Check if the token is known and the request is GET.
      if (_keyMap.containsKey(token) && request.method == 'GET') {
        // Create a request for the real, encrypted, media.
        final response = await _client.send(
          await Request(
            'GET',
            request.uri.replace(
              scheme: 'https',
              host: _cdnHostMap[token],
              port: HttpClient.defaultHttpsPort,
            ),
          ),
        );

        // Set some HTTP properties.
        request.response.headers.contentType = _contentType;
        request.response.contentLength = response.contentLength;
        request.response.statusCode = response.statusCode;

        // Create the decoder, which can decrypt the encrypted data stream.
        final decoder = XORDecoder(base64Decode(_keyMap[token]));

        // Map the encrypted data stream to a decrypted stream, and pipe it
        // to the response of the original HTTP request.
        decoder.mapStream(response.stream).pipe(request.response);
      } else {
        // If it's not either of those things, 404.
        request.response.statusCode = HttpStatus.notFound;
        request.response.contentLength = 0;
        request.response.close();
      }
    });
  }

  // Closes the server.
  Future stop() => _server.close();
}

[ryanheise]

I'm using your proxy suggestion for now, and it works really well - thanks for that. The one problem is that the server is accessible with a web browser, which isn't ideal (especially as this method is used in this generic plugin) as private data can be exposed.

Do you know of any way to remedy that?

My first thought was to add an Authorization header where only this plugin would know the token, but then you would need to get the platform side to pass in the header, which is fine for ExoPlayer but there is no equivalent for iOS.

My second thought is that you can just embed the authorization token in the URL which should be fine on both platforms.

[hacker1024]

I didn't think of that second option - thanks again.

[ryanheise]

I have just created a new branch called proxy_improvements which provides a new StreamAudioSource that allows streaming data dynamically and supports range requests. To use it, you simply create a subclass and override two methods:

class MyAudioSource extends StreamAudioSource {
  /// Used by the player to read a byte range of encoded audio data in small
  /// chunks, from byte position [start] inclusive (or from the beginning of the
  /// audio data if not specified) to [end] exclusive (or the end of the audio
  /// data if not specified).
  @override
  Stream<List<int>> read([int start, int end]) {
    ...
  }

  /// Used by the player to determine the total number of bytes of data in this
  /// audio source.
  @override
  int get lengthInBytes {
    return ...;
  }
}

This is part of a larger effort (see #272 ) to refactor the proxy to provide various other functions other than just headers. For example, the proxy is now also used to serve assets directly from the asset bundle rather than writing them to file first.

I have not implemented any security token yet, but that is certainly a good idea to implement, too.

[ryanheise]

In the latest versions of Android, apps using the proxy will get the "clear text" error since the proxy uses HTTP rather than HTTPS. The proxy can be made to use HTTPS by using bindSecure instead of bind, but it is complicated by the fact that Android does not work well with self-signed certificates:

https://stackoverflow.com/questions/38217551/how-can-i-use-an-avplayer-with-https-and-self-signed-server-certificates

There would be very little benefit in using HTTPS with even an issued certificate because the certificate would be stored within the app bundle and could be easily extracted by a hacker. The only benefit would seem to be that an Android app that does not want to enable clear-text for any reason will be able to remove the one and only need for it by using bindSecure.

A more secure approach may end up requiring platform-specific code.

[ryanheise]

Also see: https://stackpointer.io/mobile/android-sniff-http-https-traffic-without-root/490/

[hacker1024]

but it is complicated by the fact that Android does not work well with self-signed certificates:

Are you sure about that? It looks like an additional certificate can be stored in the raw resource directory and added to the AndroidManifest.xml.

Regardless, it's fairly trivial to enable cleartext traffic (hopefully localhost or 127.0.0.1 will work here, otherwise it can be enabled completely).

[ryanheise]

Sorry, I meant iOS for the first point (I'm aware Android works).

For the second point, thanks for pointing me to that link. I should update the current instructions which show how to enable cleartext completely, and include a note on how you can selectively enable it only for the loopback address. That will at least help to address people's concerns about having to enable cleartext globally.

What remains then is to figure out what would be an appropriate level of security to protect the proxy. For now, my current idea is to randomly generate a token on plugin initialisation using by default a hash of various random and internal values which can be found inside the app's process, and then to provide an override method for apps that want to set their own token.

I don't know if it is worth creating a separate token for each URL (if the hacker can crack one, they can crack the others), but since you're using a per-URL token, maybe you have some thoughts here?

[hacker1024]

If the same token is used for every URL, someone with a RAM viewer could theoretically find it and use it for every URL. If a unique token is generated for every URL, it would be much harder to crack as not only the RAM must be analysed but also the generation code needs to be reverse-engineered to predict upcoming tokens.

So IMHO per-URL tokens seem significantly more secure.

If it adds too much complexity to implement, however, it may not be worth it - after all, what kind of audio could possibly be so important to protect in this way? The user can always use a recorder app...

[ryanheise]

Yeah, someone willing to go to the extent of using a RAM viewer would have been able to find easier ways to discover the token, such as an HTTP sniffer. I guess you could have a rotating key, etc., but I'd rather implement something simple and then perhaps spend my time looking into other ways to stream the data from Dart to the platform besides an exposed port.

It would require some platform specific code, but there would be benefits to sending ByteBuffer data through the platform channel.

[swiftymf]

Sorry, I meant iOS for the first point (I'm aware Android works).

For the second point, thanks for pointing me to that link. I should update the current instructions which show how to enable cleartext completely, and include a note on how you can selectively enable it only for the loopback address. That will at least help to address people's concerns about having to enable cleartext globally.

What remains then is to figure out what would be an appropriate level of security to protect the proxy. For now, my current idea is to randomly generate a token on plugin initialisation using by default a hash of various random and internal values which can be found inside the app's process, and then to provide an override method for apps that want to set their own token.

I don't know if it is worth creating a separate token for each URL (if the hacker can crack one, they can crack the others), but since you're using a per-URL token, maybe you have some thoughts here?

@ryanheise I know this issue is closed, but was wondering if there's a quick answer. I'm setting the userAgent when initializing the AudioPlayer and is cause the same clearText error. I'd prefer not to globally set that to true. Is there a way to set it to true only for that request?

[ryanheise]

Hi @swiftymf , I don't know of any way to conditionally set this. I don't know if this is the case for your app, but maybe you could write your own app logic to just not make any other HTTP requests for non-https URLs. If all you need is to set the user agent header, that's something that eventually could be implemented in such a way that doesn't require a proxy, but it's not a priority, unless someone were to submit a pull request. Actually, the Android side has always been easy to do this, but the iOS side either involves undocumented APIs which should be avoided, or roundabout tricky code.