rust-lang / flate2-rs

DEFLATE, gzip, and zlib bindings for Rust
https://docs.rs/flate2
Apache License 2.0
891 stars 158 forks source link

Is it Flate2 slow? #259

Closed TheYkk closed 3 years ago

TheYkk commented 3 years ago

I am trying to compress ~100Mb file. But it compress 100mb file In 50 seconds. I wonder if the library is slow or if there's an error in the code I wrote.

The output is

➜ cargo run
   Compiling logwatcher v0.1.1 (/home/kaan/Work/projects/test-oss/logwatcher)
    Finished dev [unoptimized + debuginfo] target(s) in 0.41s
     Running `target/debug/logwatcher`
Rand OK 76.752903ms
File OK 76.821752ms
Gzip OK 77.209878ms
Write OK 53.218462772s
Fnish OK 53.22517111s

Rust version

➜ rustc --version
rustc 1.50.0-nightly (f76ecd066 2020-12-15)

Cargo.toml

[package]
name = "logwatcher"
authors = ["Kaan Karakaya"]
edition = "2018"

[dependencies]
flate2 = "1.0"

Rust code

use flate2::write::GzEncoder;
use flate2::Compression;
use std::fs::File;
use std::io::prelude::*;

use std::fs;
use std::time::Instant;

fn main() -> std::io::Result<()> {
    let start = Instant::now();

    // ? deaa file is 109M file
    let con = fs::read_to_string("deaa")?;
    println!("Rand OK {:?}", start.elapsed());

    let output = File::create("den.gz").unwrap();
    println!("File OK {:?}", start.elapsed());

    // ? Gzip
    let mut gzip = GzEncoder::new(output, Compression::fast());
    println!("Gzip OK {:?}", start.elapsed());

    // ! Slow: 53 seconds for 109 mb
    gzip.write_all(con.as_bytes()).unwrap();
    println!("Write OK {:?}", start.elapsed());
    gzip.finish().unwrap();
    println!("Fnish OK {:?}", start.elapsed());

    Ok(())
}
TheYkk commented 3 years ago

I found the issue if I run rust with cargo run it takes 50 seconds+. But if I build with release option it takes 1 second