TranscodingStreams.jl
TranscodingStreams.jl is a package for transcoding data streams, which is:
- fast: small overhead and specialized methods,
- consistent: basic I/O operations you already know will work as you expect,
- generic: support any I/O objects like files, buffers, pipes, etc., and
- extensible: you can define a new codec to transcode data.
Installation
Pkg.add("TranscodingStreams")Installing a codec package will install TranscodingStreams.jl as well, and so in general you don't need to explicitly install it.
Usage
using TranscodingStreams, CodecZlib
# Some text.
text = """
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sollicitudin
mauris non nisi consectetur, a dapibus urna pretium. Vestibulum non posuere
erat. Donec luctus a turpis eget aliquet. Cras tristique iaculis ex, eu
malesuada sem interdum sed. Vestibulum ante ipsum primis in faucibus orci luctus
et ultrices posuere cubilia Curae; Etiam volutpat, risus nec gravida ultricies,
erat ex bibendum ipsum, sed varius ipsum ipsum vitae dui.
"""
# Streaming API.
stream = IOBuffer(text)
stream = TranscodingStream(GzipCompressor(), stream)
stream = TranscodingStream(GzipDecompressor(), stream)
for line in eachline(stream)
println(line)
end
close(stream)
# Array API.
array = Vector{UInt8}(text)
array = transcode(GzipCompressor, array)
array = transcode(GzipDecompressor, array)
@assert text == String(array)
# Pre-allocated API
import TranscodingStreams: Buffer
compressed_buffer = Buffer(Vector{UInt8}(undef, ncodeunits(text)))
decompressed_buffer = Buffer(codeunits(text))
transcode(GzipCompressor, decompressed_buffer, compressed_buffer)
transcode(GzipDecompressor, compressed_buffer, decompressed_buffer)
@assert text == String(decompressed_buffer.data)Each codec has an alias to its transcoding stream type for ease of use. For
example, GzipCompressorStream{S} = TranscodingStream{GzipCompressor,S} where S<:IO.
Consult the docs for more details and examples.
Codec packages
TranscodingStreams.jl offers I/O interfaces to users. It also offers a protocol
suite to communicate with various codecs. However, specific codecs are not
included in this package except the Noop codec, which does nothing to data.
The user need to install codecs as a plug-in to do something meaningful.
The following codec packages support the protocol suite: