using either lz4 or bz2 does not cause any perf hit
using gz hits perf enormously (factor of 4x or more) because gzip decompression can only be done serially (so it is very slow when we read files one at a time and results in most cores being unused); users who insist on using gz for some mind boggling reason should deal with this by running multiple gtpro processes; one process per every 4-6 cpu cores ought to do it
also compress output in format matching input
using either lz4 or bz2 does not cause any perf hit
using gz hits perf enormously (factor of 4x or more) because gzip decompression can only be done serially (so it is very slow when we read files one at a time and results in most cores being unused); users who insist on using gz for some mind boggling reason should deal with this by running multiple gtpro processes; one process per every 4-6 cpu cores ought to do it