Feature Request: InputStream and OutputStream For Transformer

[nicholas-signal]

[REQUIRED] Use case description

I work on an encrypted messaging app (https://github.com/signalapp/Signal-Android) and we want to use the media3 Transformer library for transcoding media before it is sent. A hard requirement of ours is "no unencrypted data leaves memory", which we have implemented by leveraging streams to do our video processing in memory without writing anything to disk. We would like to adapt our existing in-memory pipeline to use media3 rather than interfacing with MediaCodec directly.

Proposed solution

API support for the use of Transformer with arbitrary InputStreams/OutputStreams, including streams such as (or conceptually similar to) CipherInputStream and CipherOutputStream.

Alternatives considered

Custom output: Forking InAppMuxer to be able to be constructed with a specific FileOutputStream (and ignore the path) and use its factory as a builder param for Transformer. Custom input: Duplicating DefaultDecoderFactory (which is package private) in order to be able to in the public constructor for DefaultAssetLoaderFactory to be used with a custom MediaSource.Factory but a default Codec.DecoderFactory.

[andrewlewis]

Thanks for the feature request!

On the input side: under alternatives considered, regarding DefaultDecoderFactory, this is visible as of 81fc9ea5094830e54c079e6853952436ad75cca0. After that change, can you use a custom data source factory with the media source factory to have control over data loading? (This is basically the same as customizing how ExoPlayer loads data.)

On the output side: would you want an MP4 container structure to be written to the output stream, or are you instead looking to receive the format information and samples and packetize them yourself? If it's the latter, this should already be possible by writing a custom muxer. We are planning to look at providing more control over how data gets output (for example, writing to a file descriptor) later this year, although we can't commit to it yet, so we may be able to provide an option that supports your use case as part of that work.

[tonihei]

One more comment for the input side: We don't have a InputStreamDataSource because InputStream alone isn't flexible enough for this interface. It always needs some wrapper code that can re-create the right input stream if needed. See https://github.com/google/ExoPlayer/issues/1086#issuecomment-166955403 for details (the comment is from 2015, but conceptually still applies)

[nicholas-signal]

regarding DefaultDecoderFactory, this is visible as of 81fc9ea

Oh great, looks like that's slated for the 1.2.0 release. That should be sufficient for our needs. Looking forward to it!

would you want an MP4 container structure to be written to the output stream, or are you instead looking to receive the format information and samples and packetize them yourself? If it's the latter, this should already be possible by writing a custom muxer. We are planning to look at providing more control over how data gets output (for example, writing to a file descriptor) later this year, although we can't commit to it yet, so we may be able to provide an option that supports your use case as part of that work.

Ideally the former, but workable with the latter. The overarching goal is supporting arbitrary size (greater than available RAM) inputs/outputs without writing to disk using streams, but the secondary goal is to reduce our maintenance burden by using off-the-shelf/first-party solutions where possible. So the former would get us closer to the secondary goal.

See google/ExoPlayer#1086 (comment)

Thanks for the context! The requirement for significant random read/write actually explains some of the intermittent issues I've been seeing with my POC implementation.