search

ckolivas / lrzip

Long Range Zip
http://lrzip.kolivas.org
GNU General Public License v2.0
618 stars 76 forks source link

Unable to compress with unlimited mode #189

Closed demhademha closed 3 years ago

demhademha commented 3 years ago

I am trying to tar a folder, and using the following code:

lrztar -Uzp 1 -L 9 ./Documents

however it says unable to use unlimited and stdin at the same time. I am not using stdin. Is this a bug? Regards stdin

pete4abw commented 3 years ago

lrztar pipes tar output to lrzip. You can't use STDIN and -U.

from lrztar tar c "$s" | lrzip "${p[@]}"

-L9 should be enough. Or you can try

tar --use-compress-program "lrzip [options]" -cf file.tar.lrz

Why are you using only one thread?

demhademha commented 3 years ago

lrztar pipes tar output to lrzip. You can't use STDIN and -U.

from lrztar tar c "$s" | lrzip "${p[@]}"

-L9 should be enough. Or you can try

tar --use-compress-program "lrzip [options]" -cf file.tar.lrz Thanks, this worked

Why are you using only one thread?

As per the FAQs in the README: Q: I want the absolute maximum compression I can possibly get, what do I do? A: Try the command line options "-Uzp 1 -L 9". This uses all available ram and ZPAQ compression, and even uses a compression window larger than you have ram. The -p 1 option disables multithreading which improves compression but at the expense of speed. Expect it to take many times longer.

pete4abw commented 3 years ago

Well, that part of the README is around 13 years ago. It's true that by using one thread, a chunk is not broken up into smaller pieces each of which is passed to the backend for separate compression. Some compressors use their own multi-threading. Using -U will bog down your system and all users on it. Time vs. Compression. This table may help.

New Benchmarks

ckolivas commented 3 years ago

lrztar is a convenience wrapper that uses stdin on lrzip, so it's not a bug, and is a correct warning. You need to use lrzip directly to unlock its full functionality.

demhademha commented 3 years ago

Hi, although I was able to get this to work earlier, tar --use-compress-program "lrzip -Uzp 1 -L 9" -cf offsets14.tar.lrz offsets14 I get the error: unlimited mode disabled - any way to fix this would be appreciated Thanks

ckolivas commented 3 years ago

Unlimited mode cannot work on piped data. It has to be used on a standalone file.

demhademha commented 3 years ago

Unlimited mode cannot work on piped data. It has to be used on a standalone file. using my example, is there a way to pipe it through lrzip?

ckolivas commented 3 years ago

Not with unlimited mode; it simply cannot work on STDIO. You have to do it in two steps - create a solid tar, then lrzip it.