luizirber / niffler

Simple and transparent support for compressed files.
Apache License 2.0
75 stars 7 forks source link

Implement `Send` encoder/decoder for Zstd #58

Closed jdidion closed 1 year ago

jdidion commented 2 years ago

Encoder and Decoder both implement Send

luizirber commented 2 years ago

Almost... the wrinkle here is that Encoder requires an explicit call to .finish(), which is currently managed by calling .auto_finish() for the basic encoder. But that uses an FnMut internally, which creates issues:

error[E0277]: `(dyn FnMut(Result<Box<dyn std::io::Write + Send>, std::io::Error>) + 'static)` cannot be sent between threads safely
   --> src/send/compression.rs:124:16
    |
124 |               Ok(Box::new(zstd::stream::write::Encoder::new(
    |  _____________--_^
    | |             |
    | |             required by a bound introduced by this call
125 | |                         out,
126 | |                         level.into(),
127 | |             )?.auto_finish()))
    | |_____________________________^ `(dyn FnMut(Result<Box<dyn std::io::Write + Send>, std::io::Error>) + 'static)` cannot be sent between threads safely
    |
    = help: the trait `Send` is not implemented for `(dyn FnMut(Result<Box<dyn std::io::Write + Send>, std::io::Error>) + 'static)`
    = note: required because of the requirements on the impl of `Send` for `Unique<(dyn FnMut(Result<Box<dyn std::io::Write + Send>, std::io::Error>) + 'static)>`
    = note: required because it appears within the type `Box<(dyn FnMut(Result<Box<dyn std::io::Write + Send>, std::io::Error>) + 'static)>`
    = note: required because it appears within the type `Option<Box<(dyn FnMut(Result<Box<dyn std::io::Write + Send>, std::io::Error>) + 'static)>>`
    = note: required because it appears within the type `AutoFinishEncoder<'_, Box<dyn std::io::Write + Send>>`
    = note: required for the cast to the object type `dyn std::io::Write + Send`

I opened https://github.com/luizirber/niffler/pull/59 to work on it (the Decoder part works)