madler / pigz

A parallel implementation of gzip for modern multi-processor, multi-core machines.
http://zlib.net/pigz/
2.63k stars 177 forks source link

Pipe input on Windows? #121

Closed duzenko closed 1 month ago

duzenko commented 1 month ago

The help message says "pigz does what gzip does" but on MINGW64 it's not the case

It does not seem to pick up input from a pipe

a@DESKTOP-OCHDMBI MINGW64 ~
$ time cat </dev/urandom | head -c 100M | gzip >/dev/null

real    0m2.041s
user    0m2.185s
sys     0m0.389s

a@DESKTOP-OCHDMBI MINGW64 ~
$ time cat </dev/urandom | head -c 100M | pigz >/dev/null
Usage: pigz [options] [files ...]
  will compress files in place, adding the suffix '.gz'.  If no files are
  specified, stdin will be compressed to stdout.  pigz does what gzip does,
  but spreads the work over multiple processors and cores when compressing.

Options:
  -0 to -9, -11        Compression level (11 is much slower, a few % better)
  --fast, --best       Compression levels 1 and 9 respectively
  -b, --blocksize mmm  Set compression block size to mmmK (default 128K)
  -c, --stdout         Write all processed output to stdout (won't delete)
  -d, --decompress     Decompress the compressed input
  -f, --force          Force overwrite, compress .gz, links, and to terminal
  -F  --first          Do iterations first, before block split for -11
  -h, --help           Display a help screen and quit
  -i, --independent    Compress blocks independently for damage recovery
  -I, --iterations n   Number of iterations for -11 optimization
  -k, --keep           Do not delete original file after processing
  -K, --zip            Compress to PKWare zip (.zip) single entry format
  -l, --list           List the contents of the compressed input
  -L, --license        Display the pigz license and quit
  -M, --maxsplits n    Maximum number of split blocks for -11
  -n, --no-name        Do not store or restore file name in/from header
  -N, --name           Store/restore file name and mod time in/from header
  -O  --oneblock       Do not split into smaller blocks for -11
  -p, --processes n    Allow up to n compression threads (default is the
                       number of online processors, or 8 if unknown)
  -q, --quiet          Print no messages, even on error
  -r, --recursive      Process the contents of all subdirectories
  -R, --rsyncable      Input-determined block locations for rsync
  -S, --suffix .sss    Use suffix .sss instead of .gz (for compression)
  -t, --test           Test the integrity of the compressed input
  -T, --no-time        Do not store or restore mod time in/from header
  -v, --verbose        Provide more verbose output
  -V  --version        Show the version of pigz
  -z, --zlib           Compress to zlib (.zz) instead of gzip format
  --                   All arguments after "--" are treated as files

real    0m0.109s
user    0m0.015s
sys     0m0.015s

a@DESKTOP-OCHDMBI MINGW64 ~
madler commented 1 month ago

It works everywhere else. This is a bug in MinGW's implementation of isatty(). See https://github.com/mintty/mintty/issues/56

You should be able to workaround the MinGW bug by providing an option on the command line, really any option (except -h). -c would be a good one, since pigz would be writing the output to stdout anyway. -- should work as well.

duzenko commented 1 month ago

FWIW -c fails with pigz: abort: trying to write compressed data to a terminal (use -f to force) -cf does work

OTOH why does gzip work even with the bug? Does MINGW ship their own custom version of gzip?

madler commented 1 month ago

Thanks for the info. I'm glad you got it to work.

I'm seeing the same usage of isatty() in the gzip source code, so I don't know why it works in this case.