search

JuliaIO / TranscodingStreams.jl

Simple, consistent interfaces for any codec.
https://juliaio.github.io/TranscodingStreams.jl/
Other
86 stars 28 forks source link

Allow extra keyword arguments in the Transcoding protocol #215

Closed nhz2 closed 2 months ago

nhz2 commented 5 months ago

This PR adds the ability to extend the Transcoding protocol in a non-breaking way.

This is desirable to support non-streaming codecs better. Ref: https://github.com/JuliaIO/CodecZstd.jl/pull/46

To do this I added two new optional functions to the protocol: process2 and startproc2. These are equivalent to their previous versions, except they can accept any keyword arguments.

These are optional to maintain compatibility with existing codecs.

~~This PR also sets the mode to :transcode instead of :write when calling startproc in unsafe_transcode! This is to allow non-streaming codecs to error if they are used in a streaming context. Ref: https://github.com/JuliaIO/CodecZstd.jl/pull/46~~

Edit: I'm removing the :transcode mode change because this can be a different PR.

mkitti commented 5 months ago

Can you give me an example of how this would work?

nhz2 commented 5 months ago

I'm working on a CodecCBlosc1 to see how new keyword arguments could be used.

The goal is to allow a compressor Codec to avoid buffering input if there is no more input in a frame and enough output space is available.

One thing I found is that any input passed to process needs to first get passed to minoutsize, so I'll need a minoutsize2 as well.