Closed Sanmayce closed 6 years ago
The second testdatafile under scrutiny is 'TEXTFILES.COM_(58096_files).tar' - one excellent representative of Textual_Madness series, his maintainer Jason is kinda keeper of Pre-Internet i.e. BBS times text documents - a hardcore textkeeper, almost crazy as me/I.
After downloading all the directories, decompressing them, and tarring them into a single file, the resultant corpus became 1.3GB strong - suitable to steadily measure "cruise speed" i.e. uncached decompression rate across a chubby chunk of RAM.
Matt Mahoney's fv tool and my Leprechaun Building-Blocks ripper come into play. Seeing that 64-bytes-chunk have entropy around 0.8 (suggests deduplicativeness is good) it would be interesting to give Zstd/7zip/BSC a go, with 31bit window, that is, all hands on deck in order to capture wind as fully as possible.
Entropy of Alphabetical Orders Table, in short, Decomposition Table, or, B.B.s Table:
--------------------------------------------------------------------------------------------------------------------
| #1 Filesize | #2 | #3 Granularity | #4 N.U.B.B.s | #5 U.B.B.s | #6 U.B.B.s_Alphabet | #7 Entropy | #8 Filename
| 001,382,122,496 | 001 | 008bit | 001,382,122,496 | 000,000,000,256 | NewOrder = 008bit | 0.000,000,185 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 002 | 016bit | 001,382,122,495 | 000,000,065,536 | NewOrder = 016bit | 0.000,047,416 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 004 | 032bit | 001,382,122,493 | 000,018,815,000 | NewOrder = 025bit | 0.013,613,121 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 008 | 064bit | 001,382,122,489 | 000,142,060,792 | NewOrder = 028bit | 0.102,784,520 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 064 | 512bit | 001,382,122,433 | 001,115,907,301 | NewOrder = 031bit | 0.807,386,688 | TEXTFILES.COM_(58096_files).tar
--------------------------------------------------------------------------------------------------------------------
Table Legend:
#1: Filesize (in bytes)
#2: Building-Blocks_Order = B.B.s_Order (in bytes)
#3: Granularity = Alphabet_Order = "Letters"_size = B.B.s_Order x 8bit
#4: N.U.B.B.s = Non-Unique-Building-Blocks = (Filesize - B.B.s_Order + 1)
#5: U.B.B.s = Unique-Building-Blocks
#6: Shrunk_Alphabet (to house U.B.B.s) = NewOrder, for (NewOrder=1; UBBs > ((1LL)<<NewOrder); NewOrder++);
#7: Entropy_Alphabetical = U.B.B.s / N.U.B.B.s; LOWER -> MORE COMPRESSIBLE; The bigger the number the higher the "diffusion" i.e.
richness of combinations, when 1 then full coverage of Entropy spectrum is present, lower values suggest "textuality",
higher Alphabetical Orders could "warn" for duplicative fragments
#8: Filename
H:\smashshop_2018-Feb-13_Judaica>lzbench173 -c4 -i1,15 -o3 -ebrotli24,1,5,11/tornado,16/blosclz,9/brieflz/crush,2/csc,5/density,3/fastlz,2/gipfeli/zstd24,12,22/zstd24LDM,12,22/lzo1b,999/lzham,4/lzham24,4/libdeflate,1,12/lz4fast,1,99/lz4/lz4hc,10,12/lizard,19,29,39,49/lzf,1/lzfse/lzg,9/lzham,1/lzjb/lzlib,9/lzma,9/lzrw,5/lzsse2,17/lzsse4,17/lzsse8,17/lzvn/pithy,9/quicklz,3/snappy/slz_zlib,3/ucl_nrv2b,9/ucl_nrv2d,9/ucl_nrv2e,9/xpack,9/xz,9/yalz77,12/yappy,99/zlib,1,5,9/zling,4/shrinker/wflz/lzmat "TEXTFILES.COM_(58096_files).tar"
lzbench 1.7.3 (64-bit Windows) Assembled by P.Skibinski
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
memcpy 6332 MB/s 6327 MB/s 1382122496 1382122496 100.00 TEXTFILES.COM_(58096_files).tar
...
The results sorted by column number 4:
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
lzma 16.04 -9 1.01 MB/s 74 MB/s 1382122496 294365747 21.30 TEXTFILES.COM_(58096_files).tar
xz 5.2.3 -9 1.06 MB/s 67 MB/s 1382122496 294368247 21.30 TEXTFILES.COM_(58096_files).tar
csc 2016-10-13 -5 2.23 MB/s 55 MB/s 1382122496 297190237 21.50 TEXTFILES.COM_(58096_files).tar
lzlib 1.8 -9 0.95 MB/s 49 MB/s 1382122496 298304742 21.58 TEXTFILES.COM_(58096_files).tar
lzham 1.0 -d26 -4 0.72 MB/s 156 MB/s 1382122496 300823143 21.77 TEXTFILES.COM_(58096_files).tar
tornado 0.6a -16 1.00 MB/s 149 MB/s 1382122496 302743656 21.90 TEXTFILES.COM_(58096_files).tar
brotli24 2017-12-12 -11 0.35 MB/s 230 MB/s 1382122496 305731677 22.12 TEXTFILES.COM_(58096_files).tar
lzham24 1.0 -4 0.86 MB/s 157 MB/s 1382122496 313141233 22.66 TEXTFILES.COM_(58096_files).tar
zstd24 1.3.3 -22 1.30 MB/s 431 MB/s 1382122496 313416488 22.68 TEXTFILES.COM_(58096_files).tar
zstd24LDM 1.3.3 -22 1.45 MB/s 435 MB/s 1382122496 319352192 23.11 TEXTFILES.COM_(58096_files).tar
lzham 1.0 -d26 -1 1.78 MB/s 159 MB/s 1382122496 344399202 24.92 TEXTFILES.COM_(58096_files).tar
zling 2016-01-10 -4 26 MB/s 135 MB/s 1382122496 362251309 26.21 TEXTFILES.COM_(58096_files).tar
zstd24 1.3.3 -12 7.24 MB/s 452 MB/s 1382122496 369501905 26.73 TEXTFILES.COM_(58096_files).tar
zstd24LDM 1.3.3 -12 7.03 MB/s 453 MB/s 1382122496 369768475 26.75 TEXTFILES.COM_(58096_files).tar
brotli24 2017-12-12 -5 16 MB/s 290 MB/s 1382122496 376186414 27.22 TEXTFILES.COM_(58096_files).tar
crush 1.0 -2 0.27 MB/s 263 MB/s 1382122496 385559339 27.90 TEXTFILES.COM_(58096_files).tar
xpack 2016-06-02 -9 9.93 MB/s 509 MB/s 1382122496 416966890 30.17 TEXTFILES.COM_(58096_files).tar
lizard 1.0 -49 1.00 MB/s 1043 MB/s 1382122496 434488576 31.44 TEXTFILES.COM_(58096_files).tar
lizard 1.0 -29 1.01 MB/s 1206 MB/s 1382122496 438313614 31.71 TEXTFILES.COM_(58096_files).tar
libdeflate 0.7 -12 4.67 MB/s 439 MB/s 1382122496 440842196 31.90 TEXTFILES.COM_(58096_files).tar
lzfse 2017-03-08 40 MB/s 422 MB/s 1382122496 443281031 32.07 TEXTFILES.COM_(58096_files).tar
zlib 1.2.11 -9 9.54 MB/s 226 MB/s 1382122496 459948910 33.28 TEXTFILES.COM_(58096_files).tar
ucl_nrv2e 1.03 -9 1.11 MB/s 222 MB/s 1382122496 466328760 33.74 TEXTFILES.COM_(58096_files).tar
zlib 1.2.11 -5 24 MB/s 222 MB/s 1382122496 468695334 33.91 TEXTFILES.COM_(58096_files).tar
ucl_nrv2d 1.03 -9 1.12 MB/s 223 MB/s 1382122496 470973152 34.08 TEXTFILES.COM_(58096_files).tar
brotli24 2017-12-12 -1 122 MB/s 243 MB/s 1382122496 474946245 34.36 TEXTFILES.COM_(58096_files).tar
lzsse2 2016-05-14 -17 0.80 MB/s 2089 MB/s 1382122496 478090440 34.59 TEXTFILES.COM_(58096_files).tar
ucl_nrv2b 1.03 -9 1.18 MB/s 225 MB/s 1382122496 479484131 34.69 TEXTFILES.COM_(58096_files).tar
lzsse4 2016-05-14 -17 0.53 MB/s 2170 MB/s 1382122496 480734209 34.78 TEXTFILES.COM_(58096_files).tar
lzsse8 2016-05-14 -17 0.48 MB/s 2135 MB/s 1382122496 483097517 34.95 TEXTFILES.COM_(58096_files).tar
libdeflate 0.7 -1 99 MB/s 402 MB/s 1382122496 502340533 36.35 TEXTFILES.COM_(58096_files).tar
lzo1b 2.09 -999 9.65 MB/s 479 MB/s 1382122496 506062307 36.61 TEXTFILES.COM_(58096_files).tar
lzmat 1.01 19 MB/s 225 MB/s 1382122496 511241291 36.99 TEXTFILES.COM_(58096_files).tar
brieflz 1.1.0 88 MB/s 131 MB/s 1382122496 514506377 37.23 TEXTFILES.COM_(58096_files).tar
lz4hc 1.8.0 -12 4.00 MB/s 1834 MB/s 1382122496 518609982 37.52 TEXTFILES.COM_(58096_files).tar
lizard 1.0 -19 2.60 MB/s 2023 MB/s 1382122496 520937114 37.69 TEXTFILES.COM_(58096_files).tar
lz4hc 1.8.0 -10 15 MB/s 1836 MB/s 1382122496 522394279 37.80 TEXTFILES.COM_(58096_files).tar
yalz77 2015-09-19 -12 13 MB/s 226 MB/s 1382122496 524148698 37.92 TEXTFILES.COM_(58096_files).tar
lizard 1.0 -39 2.58 MB/s 1869 MB/s 1382122496 531515792 38.46 TEXTFILES.COM_(58096_files).tar
lzg 1.0.8 -9 0.49 MB/s 447 MB/s 1382122496 534692404 38.69 TEXTFILES.COM_(58096_files).tar
zlib 1.2.11 -1 58 MB/s 211 MB/s 1382122496 539734064 39.05 TEXTFILES.COM_(58096_files).tar
lzvn 2017-03-08 32 MB/s 607 MB/s 1382122496 564298861 40.83 TEXTFILES.COM_(58096_files).tar
quicklz 1.5.0 -3 35 MB/s 606 MB/s 1382122496 566710303 41.00 TEXTFILES.COM_(58096_files).tar
gipfeli 2016-07-13 165 MB/s 283 MB/s 1382122496 580927128 42.03 TEXTFILES.COM_(58096_files).tar
pithy 2011-12-24 -9 196 MB/s 983 MB/s 1382122496 596838772 43.18 TEXTFILES.COM_(58096_files).tar
density 0.12.5 beta -3 200 MB/s 212 MB/s 1382122496 600009694 43.41 TEXTFILES.COM_(58096_files).tar
lzrw 15-Jul-1991 -5 89 MB/s 336 MB/s 1382122496 616934562 44.64 TEXTFILES.COM_(58096_files).tar
fastlz 0.1 -2 190 MB/s 344 MB/s 1382122496 685175128 49.57 TEXTFILES.COM_(58096_files).tar
slz_zlib 1.0.0 -3 136 MB/s 181 MB/s 1382122496 692113550 50.08 TEXTFILES.COM_(58096_files).tar
yappy 2014-03-22 -99 52 MB/s 1485 MB/s 1382122496 694938661 50.28 TEXTFILES.COM_(58096_files).tar
lzf 3.6 -1 189 MB/s 385 MB/s 1382122496 697137558 50.44 TEXTFILES.COM_(58096_files).tar
blosclz 2015-11-10 -9 163 MB/s 541 MB/s 1382122496 705699590 51.06 TEXTFILES.COM_(58096_files).tar
lz4fast 1.8.0 -1 301 MB/s 1874 MB/s 1382122496 722299465 52.26 TEXTFILES.COM_(58096_files).tar
lz4 1.8.0 306 MB/s 1881 MB/s 1382122496 722299465 52.26 TEXTFILES.COM_(58096_files).tar
snappy 1.1.4 181 MB/s 729 MB/s 1382122496 724378872 52.41 TEXTFILES.COM_(58096_files).tar
wflz 2015-09-16 154 MB/s 539 MB/s 1382122496 811018853 58.68 TEXTFILES.COM_(58096_files).tar
lzjb 2010 162 MB/s 306 MB/s 1382122496 899327706 65.07 TEXTFILES.COM_(58096_files).tar
lz4fast 1.8.0 -99 2493 MB/s 4551 MB/s 1382122496 1246929570 90.22 TEXTFILES.COM_(58096_files).tar
shrinker 0.1 102 MB/s 4924 MB/s 1382122496 1361132906 98.48 TEXTFILES.COM_(58096_files).tar
H:\smashshop_2018-Feb-13_Judaica>"turbobenchs_Official_v17.04_-_build_07_Apr_2017.exe" "TEXTFILES.COM_(58096_files).tar" -elibdeflate,12/oodle,19,49,89,112,114,116,118,129/lzsse2,17/lzturbo,19,12,29,22,39,32,49,59/zstd,12,22/lizard,19,29,39,49/brotli,11/lzma,9/bzip2/xpack,9/chameleon,2/density,3/lzham,4/trle/bsc,3,6/zpaq,2,5 -g -I1 -J31 -k1 -B2G
...
356885471 25.8 10.60 23.13 bzip2 TEXTFILES.COM_(58096_files).tar
**ERROR at 731148480:3c, 20**
353383914 25.6 0.43 428.70 ?xpack 9 TEXTFILES.COM_(58096_files).tar
...
TurboBench: - Sat Dec 02 15:49:06 2017
C Size ratio% C MB/s D MB/s Name File
250105557 18.1 0.31 0.30 zpaq 5 TEXTFILES.COM_(58096_files).tar.tbb
270967780 19.6 0.51 83.00 lzturbo 49 TEXTFILES.COM_(58096_files).tar.tbb
273561108 19.8 5.69 12.60 lzturbo 59 TEXTFILES.COM_(58096_files).tar.tbb
277187778 20.1 13.64 6.34 bsc 6 TEXTFILES.COM_(58096_files).tar.tbb
283683525 20.5 0.59 404.32 lzturbo 39 TEXTFILES.COM_(58096_files).tar.tbb
286204548 20.7 0.64 155.45 lzham 4 TEXTFILES.COM_(58096_files).tar.tbb
292753766 21.2 0.25 437.20 oodle 89 TEXTFILES.COM_(58096_files).tar.tbb
292754261 21.2 0.23 437.19 oodle 129 TEXTFILES.COM_(58096_files).tar.tbb
294785080 21.3 1.01 74.55 lzma 9 TEXTFILES.COM_(58096_files).tar.tbb
297798812 21.5 0.93 421.50 zstd 22 TEXTFILES.COM_(58096_files).tar.tbb
300158804 21.7 0.17 299.12 oodle 19 TEXTFILES.COM_(58096_files).tar.tbb
322566854 23.3 0.37 266.17 brotli 11 TEXTFILES.COM_(58096_files).tar.tbb
326268013 23.6 0.62 631.53 lzturbo 29 TEXTFILES.COM_(58096_files).tar.tbb
327812776 23.7 19.09 12.63 bsc 3 TEXTFILES.COM_(58096_files).tar.tbb
337774878 24.4 21.63 441.86 lzturbo 32 TEXTFILES.COM_(58096_files).tar.tbb
353383914 25.6 0.43 428.70 ?xpack 9 TEXTFILES.COM_(58096_files).tar.tbb
356885471 25.8 10.60 23.13 bzip2 TEXTFILES.COM_(58096_files).tar.tbb
369533604 26.7 7.13 508.55 zstd 12 TEXTFILES.COM_(58096_files).tar.tbb
385423296 27.9 3.20 60.41 zpaq 2 TEXTFILES.COM_(58096_files).tar.tbb
429378865 31.1 0.25 2236.73 oodle 116 TEXTFILES.COM_(58096_files).tar.tbb
433949943 31.4 0.26 2862.57 oodle 118 TEXTFILES.COM_(58096_files).tar.tbb
434695243 31.5 1.00 1047.15 lizard 49 TEXTFILES.COM_(58096_files).tar.tbb
438494294 31.7 1.01 1211.03 lizard 29 TEXTFILES.COM_(58096_files).tar.tbb
440842503 31.9 4.70 419.04 libdeflate 12 TEXTFILES.COM_(58096_files).tar.tbb
450558126 32.6 20.70 773.98 lzturbo 22 TEXTFILES.COM_(58096_files).tar.tbb
478092864 34.6 0.79 2086.46 lzsse2 17 TEXTFILES.COM_(58096_files).tar.tbb
490930678 35.5 11.47 2408.14 oodle 114 TEXTFILES.COM_(58096_files).tar.tbb
518918583 37.5 2.95 1893.05 oodle 49 TEXTFILES.COM_(58096_files).tar.tbb
520236948 37.6 0.63 2702.05 lzturbo 19 TEXTFILES.COM_(58096_files).tar.tbb
520939095 37.7 2.61 2021.26 lizard 19 TEXTFILES.COM_(58096_files).tar.tbb
531426412 38.5 2.60 1872.04 lizard 39 TEXTFILES.COM_(58096_files).tar.tbb
551896961 39.9 39.03 2601.60 oodle 112 TEXTFILES.COM_(58096_files).tar.tbb
557133874 40.3 35.49 2649.89 lzturbo 12 TEXTFILES.COM_(58096_files).tar.tbb
599413372 43.4 218.64 219.32 density 3 TEXTFILES.COM_(58096_files).tar.tbb
825708086 59.7 1084.83 1782.72 chameleon 2 TEXTFILES.COM_(58096_files).tar.tbb
1221989897 88.4 186.11 4710.79 trle TEXTFILES.COM_(58096_files).tar.tbb
H:\smashshop_2018-Feb-13_Judaica>call _bunchIFY.BAT "TEXTFILES.COM_(58096_files).tar"
H:\smashshop_2018-Feb-13_Judaica>sha1sum "TEXTFILES.COM_(58096_files).tar"
f8cd4ad35c75f3f40df1658cd54bdc14eb8d952e TEXTFILES.COM_(58096_files).tar
H:\smashshop_2018-Feb-13_Judaica>timer64 PPMd_varI_rev2_Intel15_32bit.exe e -o6 -m256 -f"TEXTFILES.COM_(58096_files).tar".O6.PPMd_varI "TEXTFILES.COM_(58096_files).tar"
Fast PPMII compressor for textual data, variant I, Apr 3 2016
TEXTFILES.COM_(58096_files).tar:1382122496 >300826015, 1.00 bpb, used:112.9MB,
Kernel Time = 1.107 = 0%
User Time = 279.725 = 98%
Process Time = 280.833 = 99% Virtual Memory = 258 MB
Global Time = 282.774 = 100% Physical Memory = 259 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 PPMd_varI_rev2_Intel15_32bit.exe e -o16 -m256 -f"TEXTFILES.COM_(58096_files).tar".O16.PPMd_varI "TEXTFILES.COM_(58096_files).tar"
Fast PPMII compressor for textual data, variant I, Apr 3 2016
TEXTFILES.COM_(58096_files).tar:1382122496 >284113628, 1.02 bpb, used:188.8MB,
Kernel Time = 1.154 = 0%
User Time = 436.350 = 99%
Process Time = 437.504 = 99% Virtual Memory = 258 MB
Global Time = 439.258 = 100% Physical Memory = 259 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "7za_v16.04_x64.exe" a -tgzip -mx9 "TEXTFILES.COM_(58096_files).tar".MX9.zip "TEXTFILES.COM_(58096_files).tar"
7-Zip (a) [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
Scanning the drive:
1 file, 1382122496 bytes (1319 MiB)
Creating archive: TEXTFILES.COM_(58096_files).tar.MX9.zip
Items to compress: 1
Files read from disk: 1
Archive size: 440531838 bytes (421 MiB)
Everything is Ok
Kernel Time = 0.686 = 0%
User Time = 1563.910 = 99%
Process Time = 1564.596 = 99% Virtual Memory = 6 MB
Global Time = 1567.239 = 100% Physical Memory = 8 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "7za_v16.04_x64.exe" a -t7z -mx9 -md=29 "TEXTFILES.COM_(58096_files).tar".MX9Dict512.7z "TEXTFILES.COM_(58096_files).tar"
7-Zip (a) [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
Scanning the drive:
1 file, 1382122496 bytes (1319 MiB)
Creating archive: TEXTFILES.COM_(58096_files).tar.MX9Dict512.7z
Items to compress: 1
Files read from disk: 1
Archive size: 284072432 bytes (271 MiB)
Everything is Ok
Kernel Time = 10.077 = 1%
User Time = 1993.630 = 237%
Process Time = 2003.708 = 239% Virtual Memory = 11299 MB
Global Time = 837.665 = 100% Physical Memory = 10483 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "xz_v5.2.3_x64.exe" -z -k -f -9 -e -v -v --lzma2=dict=512MiB --threads=1 "TEXTFILES.COM_(58096_files).tar"
xz_v5.2.3_x64: Filter chain: --lzma2=dict=512MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz_v5.2.3_x64: 5,378 MiB of memory is required. The limiter is disabled.
xz_v5.2.3_x64: Decompression will need 513 MiB of memory.
TEXTFILES.COM_(58096_files).tar (1/1)
100 % 264.8 MiB / 1,318.1 MiB = 0.201 792 KiB/s 28:24
Kernel Time = 5.522 = 0%
User Time = 1681.534 = 98%
Process Time = 1687.057 = 98% Virtual Memory = 5389 MB
Global Time = 1704.657 = 100% Physical Memory = 5390 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "bsc_v3.1.0_x64.exe" e "TEXTFILES.COM_(58096_files).tar" "TEXTFILES.COM_(58096_files).tar".ST6Block512.bsc -b512 -m6 -cp -Tt
This is bsc, Block Sorting Compressor. Version 3.1.0. 8 July 2012.
Copyright (c) 2009-2012 Ilya Grebnov <Ilya.Grebnov@gmail.com>.
TEXTFILES.COM_(58096_files).tar compressed 1382122496 into 277656610 in 108.982 seconds.
Kernel Time = 2.870 = 2%
User Time = 98.109 = 89%
Process Time = 100.979 = 92% Virtual Memory = 2407 MB
Global Time = 109.744 = 100% Physical Memory = 2401 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "zpaq_v7.05_x64.exe" add "TEXTFILES.COM_(58096_files).tar".method29.zpaq "TEXTFILES.COM_(58096_files).tar" -method 29 -threads 1
zpaq v7.05 journaling archiver, compiled Apr 17 2015
Adding 1382.122496 MB in 1 files -method 29 -threads 1 at 2017-12-03 01:25:31.
39.06% 0:00:11 [1..7637] 536740679 -method 29,171,1
78.38% 0:00:04 [7638..15443] 536851892 -method 29,176,1
100.00% 0:00:00 + TEXTFILES.COM_(58096_files).tar 1382122496 -> 1370358887
100.00% 0:00:00 [15444..19576] 296844644 -method 29,183,1
1 +added, 0 -removed.
0.000000 + (1382.122496 -> 1370.358887 -> 347.125309) = 347.125309 MB
608.950 seconds (all OK)
Kernel Time = 2.683 = 0%
User Time = 614.565 = 100%
Process Time = 617.249 = 101% Virtual Memory = 3724 MB
Global Time = 608.975 = 100% Physical Memory = 3597 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "zpaq_v7.05_x64.exe" add "TEXTFILES.COM_(58096_files).tar".method59.zpaq "TEXTFILES.COM_(58096_files).tar" -method 59 -threads 1
zpaq v7.05 journaling archiver, compiled Apr 17 2015
Adding 1382.122496 MB in 1 files -method 59 -threads 1 at 2017-12-03 01:35:41.
39.06% 0:00:11 [1..7637] 536740679 -method 59,171,1
78.38% 0:00:04 [7638..15443] 536851892 -method 59,176,1
100.00% 0:00:00 + TEXTFILES.COM_(58096_files).tar 1382122496 -> 1370358887
100.00% 0:00:00 [15444..19576] 296844644 -method 59,183,1
1 +added, 0 -removed.
0.000000 + (1382.122496 -> 1370.358887 -> 220.271375) = 220.271375 MB
5427.540 seconds (all OK)
Kernel Time = 6.130 = 0%
User Time = 5427.337 = 99%
Process Time = 5433.468 = 100% Virtual Memory = 6569 MB
Global Time = 5427.899 = 100% Physical Memory = 6416 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 lz5 -49 -B7 --no-frame-crc "TEXTFILES.COM_(58096_files).tar"
using blocks of size 262144 KB
Compressed filename will be : TEXTFILES.COM_(58096_files).tar.lz5
Compressed 1382122496 bytes into 375169989 bytes ==> 27.14%
Kernel Time = 0.951 = 0%
User Time = 1353.324 = 98%
Process Time = 1354.275 = 99% Virtual Memory = 676 MB
Global Time = 1366.995 = 100% Physical Memory = 497 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "nanozip-0.09a.win64.exe" a -t1 -cc "TEXTFILES.COM_(58096_files).tar".cc.nz "TEXTFILES.COM_(58096_files).tar"
NanoZip 0.09 alpha/Win64 (C) 2008-2011 Sami Runsas www.nanozip.net
Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz|35496 MHz|#2+HT|15764/16332 MB
Archive: TEXTFILES.COM_(58096_files).tar.cc.nz
Threads: 1, memory: 512 MB
Compressor #0: nz_cm [524 MB]
Compressed 1 382 122 496 into 227 637 734 in 27m 8.58s, 829 KB/s
IO-in: 0.83s, 1575 MB/s. IO-out: 0.14s, 1497 MB/s
Kernel Time = 3.525 = 0%
User Time = 1663.501 = 99%
Process Time = 1667.026 = 99% Virtual Memory = 539 MB
Global Time = 1668.830 = 100% Physical Memory = 525 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "nanozip-0.09a.win64.exe" a -t1 -co "TEXTFILES.COM_(58096_files).tar".co.nz "TEXTFILES.COM_(58096_files).tar"
NanoZip 0.09 alpha/Win64 (C) 2008-2011 Sami Runsas www.nanozip.net
Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz|53644 MHz|#2+HT|15759/16332 MB
Archive: TEXTFILES.COM_(58096_files).tar.co.nz
Threads: 1, memory: 512 MB
Compressor #0: nz_optimum1 [506 MB]
Compressed 1 382 122 496 into 280 316 190 in 5m 27.46s, 4122 KB/s
IO-in: 0.83s, 1586 MB/s. IO-out: 0.09s, 2700 MB/s
Kernel Time = 8.283 = 2%
User Time = 327.180 = 97%
Process Time = 335.464 = 99% Virtual Memory = 524 MB
Global Time = 336.925 = 100% Physical Memory = 516 MB
CABARC, Microsoft (R) Cabinet Tool - Version 5.1.2600.0, Copyright (c) Microsoft Corporation.
H:\smashshop_2018-Feb-13_Judaica>timer64 cabarc.exe -m LZX:21 N "TEXTFILES.COM_(58096_files).tar".LZX21.cab "TEXTFILES.COM_(58096_files).tar"
Microsoft (R) Cabinet Tool - Version 5.1.2600.0
Copyright (c) Microsoft Corporation. All rights reserved..
Creating new cabinet 'TEXTFILES.COM_(58096_files).tar.LZX21.cab' with compression 'LZX:21':
-- adding TEXTFILES.COM_(58096_files).tar
Completed successfully
Kernel Time = 3.026 = 0%
User Time = 1015.862 = 98%
Process Time = 1018.889 = 98% Virtual Memory = 20 MB
Global Time = 1034.099 = 100% Physical Memory = 22 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 cabarc.exe -m MSZIP N "TEXTFILES.COM_(58096_files).tar".MSZIP.cab "TEXTFILES.COM_(58096_files).tar"
Microsoft (R) Cabinet Tool - Version 5.1.2600.0
Copyright (c) Microsoft Corporation. All rights reserved..
Creating new cabinet 'TEXTFILES.COM_(58096_files).tar.MSZIP.cab' with compression 'MSZIP':
-- adding TEXTFILES.COM_(58096_files).tar
Completed successfully
Kernel Time = 2.308 = 0%
User Time = 218.635 = 92%
Process Time = 220.944 = 93% Virtual Memory = 1 MB
Global Time = 237.385 = 100% Physical Memory = 3 MB
Compress, version: (N)compress 4.2.4.4, compiled: Fri, Aug 23, 2013 11:56:09. Authors: Peter Jannesen, Dave Mack, Spencer W. Thomas, Jim McKie, Steve Davies, Ken Turkowski, James A. Woods, Joe Orost.
H:\smashshop_2018-Feb-13_Judaica>timer64 compress.exe -c "TEXTFILES.COM_(58096_files).tar" 1>"TEXTFILES.COM_(58096_files).tar".Z
H:\smashshop_2018-Feb-13_Judaica>timer64 zstd-v1.3.3-win64.exe --ultra -22 --zstd=wlog=29,clog=30,hlog=30,slog=26 "TEXTFILES.COM_(58096_files).tar"
TEXTFILES.COM_(58096_files).tar : 20.61% (1382122496 => 284848217 bytes, TEXTFILES.COM_(58096_files).tar.zst)
Kernel Time = 3.010 = 0%
User Time = 1671.675 = 98%
Process Time = 1674.686 = 98% Virtual Memory = 8724 MB
Global Time = 1698.245 = 100% Physical Memory = 8726 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 rz_1.01.exe a -d 512M "TEXTFILES.COM_(58096_files).tar".512M.rz "TEXTFILES.COM_(58096_files).tar"
*** RAZOR Archiver 1.01 (2017-09-14) - DEMO/TEST version ***
*** (c) Christian Martelock (christian.martelock@web.de) ***
Scanning h:\smashshop_2018-feb-13_judaica\textfiles.com_(58096_files).tar
Found 0 dirs, 1 files, 1382122496 bytes.
Creating archive TEXTFILES.COM_(58096_files).tar.512M.rz
Window : 524288K (4096M..1024G)
Header : 64
Size : 243608643
Archive ok. Added 0 dirs, 1 files, 1382122496 bytes.
CPU time = 6756.294s / wall time = 5211.298s
Kernel Time = 22.339 = 0%
User Time = 6756.294 = 129%
Process Time = 6778.633 = 130% Virtual Memory = 6491 MB
Global Time = 5211.397 = 100% Physical Memory = 6075 MB
12/02/2017 07:06 PM 220,271,375 TEXTFILES.COM_(58096_files).tar.method59.zpaq
12/02/2017 07:56 PM 227,637,734 TEXTFILES.COM_(58096_files).tar.cc.nz
12/02/2017 10:19 PM 243,608,643 TEXTFILES.COM_(58096_files).tar.512M.rz
12/02/2017 05:25 PM 277,656,610 TEXTFILES.COM_(58096_files).tar.ST6Block512.bsc
02/12/2018 05:42 PM 277,691,252 TEXTFILES.COM_(58096_files).tar.L9Dict512.xz
12/02/2017 08:02 PM 280,316,190 TEXTFILES.COM_(58096_files).tar.co.nz
12/02/2017 04:55 PM 284,072,432 TEXTFILES.COM_(58096_files).tar.MX9Dict512.7z
12/02/2017 04:15 PM 284,113,675 TEXTFILES.COM_(58096_files).tar.O16.PPMd_varI
02/12/2018 05:42 PM 284,848,217 TEXTFILES.COM_(58096_files).tar.zst
12/02/2017 04:07 PM 300,826,062 TEXTFILES.COM_(58096_files).tar.O6.PPMd_varI
12/02/2017 08:19 PM 342,191,276 TEXTFILES.COM_(58096_files).tar.LZX21.cab
12/02/2017 05:35 PM 347,125,309 TEXTFILES.COM_(58096_files).tar.method29.zpaq
02/12/2018 05:42 PM 375,169,989 TEXTFILES.COM_(58096_files).tar.M49Block256.lz5
12/02/2017 04:41 PM 440,531,838 TEXTFILES.COM_(58096_files).tar.MX9.zip
12/02/2017 08:23 PM 460,971,224 TEXTFILES.COM_(58096_files).tar.MSZIP.cab
12/02/2017 08:24 PM 616,008,949 TEXTFILES.COM_(58096_files).tar.Z
02/12/2018 05:42 PM 1,382,122,496 TEXTFILES.COM_(58096_files).tar
H:\smashshop_2018-Feb-13_Judaica>timer64 lz4_v1_8_1_win64 -12 "TEXTFILES.COM_(58096_files).tar" "TEXTFILES.COM_(58096_files).tar".12.lz4
Compressed 1382122496 bytes into 519780435 bytes ==> 37.61%
Kernel Time = 0.748 = 0%
User Time = 366.305 = 99%
Process Time = 367.054 = 99% Virtual Memory = 10 MB
Global Time = 369.120 = 100% Physical Memory = 10 MB
Next run, under way already, is 'Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar' - a collection of all (20+ languages) Bibles that I downloaded - excellent test for parsing prowess within 4MB windows, mainly. Also, seeing how LzTurbo 59 overpowers BSC, I am eager to see it also full-throttle, especially with the superindicative for textual [de]compression power corpus: INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM9(English_140479_htm_files).tar (2,037,880,832 bytes) http://www.sacred-texts.com/cdshop/dvd90/titl90.htm
Another textkeeper of highest caliber:
His corpus deserves being heavily benchmarked...
John Bruno Hare (JBH or Bruno), the founder and architect of ISTA (Internet Sacred Text Archive) passed away on April 27, 2010 after a four-year battle with Melanoma. JBH’s life mission was to keep the archive free and available worldwide, forever, and ISTA is his legacy. Bruno’s efforts placed this website, sacred-texts.com, among the top 10,000 read websites in the United States, and among the top 20,000 read websites of the entire Internet. He dedicated ISTA to religious tolerance and scholarship, calling it a “quiet place in cyberspace."
"THE INTERNET SACRED TEXT ARCHIVE DVD-ROM 9.0 has electronic texts of over 1700 of the most important books ever written, most of which were transcribed especially for sacred-texts. Years of research and scholarship went into this product: all the core texts of religion, mythology, folklore and the esoteric on one disk. For centuries these texts were stashed in dusty libraries. Now you can finally unlock the secrets of the ages on your home computer. Scholars have only dreamed of this kind of access, ..."
Supernifty corpus, indeed.
Hi Georgi, thanks for publishing your interesting results on the Turbobench repositery and for all your efforts and contributions to the data compression field.
Always, Hamid. Very glad that you as host see it useful. Really feel that having thread as this will allow people to propose files of any kind for test, thus eliminating the resentment and unhappiness. Hopefully, one day all mainstream/major performers will be benchmarked exhaustively - without any compromises, SmashShop package is a strong start/baseline. Maybe someone else will present better, richer and more refined package.
Also, in future runs will add as notes next:
Fabian Giesen @rygorous took time to point out my mistakes, here his remarks come as notes:
Note1: ... 12=Hydra (Hydra blends Kraken/Mermaid/Selkie per block as controlled by space/speed trade-off).
Note2: (Similarly you can get higher-speed decompression at lower ratio by increasing it.)
Note3: If you want higher compression, you need to lower the decode-speed target. (Controlled via spaceSpeedTradeoffBytes in the extended compressor config.)
Note4: FWIW, if you're benchmarking Oodle, compression level is _not_ the sole parameter to change to get min size. Different levels control encoder effort but they still target the same rough level of decode speed.
Note5: Compression level only goes up to 8. (0=none 1=SuperFast 2=VeryFast 3=Fast 4=Normal 5=Optimal1 .. 8=Optimal4) I think it clamps higher values but they don't do anything.
Note6: http://cbloomrants.blogspot.bg/2017/09/oodle-tuneability-with-space-speed.html
The logs for amalgam of 103 Bibles (in 40+ languages) and the ISTA come below.
The fv generated 'Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.bmp':
H:\smashshop_2018-Feb-13_Judaica>lzbench173 -c4 -i1,15 -o3 -ebrotli24,1,5,11/tornado,16/blosclz,9/brieflz/crush,2/csc,5/density,3/fastlz,2/gipfeli/zstd24,12,22/zstd24LDM,12,22/lzo1b,999/lzham,4/lzham24,4/libdeflate,1,12/lz4fast,1,99/lz4/lz4hc,10,12/lizard,19,29,39,49/lzf,1/lzfse/lzg,9/lzham,1/lzjb/lzlib,9/lzma,9/lzrw,5/lzsse2,17/lzsse4,17/lzsse8,17/lzvn/pithy,9/quicklz,3/snappy/slz_zlib,3/ucl_nrv2b,9/ucl_nrv2d,9/ucl_nrv2e,9/xpack,9/xz,9/yalz77,12/yappy,99/zlib,1,5,9/zling,4/shrinker/wflz/lzmat Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzbench 1.7.3 (64-bit Windows) Assembled by P.Skibinski
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
memcpy 6415 MB/s 6416 MB/s 630349312 630349312 100.00 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
...
The results sorted by column number 4:
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
lzlib 1.8 -9 0.97 MB/s 82 MB/s 630349312 73233416 11.62 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
xz 5.2.3 -9 1.42 MB/s 119 MB/s 630349312 73694019 11.69 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzma 16.04 -9 1.31 MB/s 134 MB/s 630349312 73714979 11.69 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
brotli24 2017-12-12 -11 0.42 MB/s 506 MB/s 630349312 74375172 11.80 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
csc 2016-10-13 -5 2.10 MB/s 105 MB/s 630349312 76509932 12.14 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zstd24 1.3.3 -22 1.05 MB/s 701 MB/s 630349312 77176177 12.24 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
tornado 0.6a -16 1.11 MB/s 291 MB/s 630349312 77524331 12.30 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzham 1.0 -d26 -4 0.88 MB/s 287 MB/s 630349312 77755762 12.34 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzham24 1.0 -4 0.92 MB/s 286 MB/s 630349312 78373249 12.43 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zstd24LDM 1.3.3 -22 1.63 MB/s 702 MB/s 630349312 79586664 12.63 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzham 1.0 -d26 -1 2.09 MB/s 219 MB/s 630349312 100498198 15.94 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lizard 1.0 -49 0.85 MB/s 1719 MB/s 630349312 103619668 16.44 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lizard 1.0 -29 0.86 MB/s 1880 MB/s 630349312 104883924 16.64 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
brotli24 2017-12-12 -5 21 MB/s 384 MB/s 630349312 125296296 19.88 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zling 2016-01-10 -4 30 MB/s 172 MB/s 630349312 131710233 20.89 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zstd24LDM 1.3.3 -12 7.62 MB/s 519 MB/s 630349312 137616216 21.83 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zstd24 1.3.3 -12 7.83 MB/s 515 MB/s 630349312 138508620 21.97 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
crush 1.0 -2 0.14 MB/s 298 MB/s 630349312 146570478 23.25 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
xpack 2016-06-02 -9 6.92 MB/s 567 MB/s 630349312 157110327 24.92 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
libdeflate 0.7 -12 3.86 MB/s 577 MB/s 630349312 160509275 25.46 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zlib 1.2.11 -9 6.32 MB/s 261 MB/s 630349312 171632959 27.23 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzsse2 2016-05-14 -17 4.14 MB/s 2569 MB/s 630349312 172562119 27.38 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzsse4 2016-05-14 -17 4.72 MB/s 2559 MB/s 630349312 173433363 27.51 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzsse8 2016-05-14 -17 4.61 MB/s 2338 MB/s 630349312 177193498 28.11 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zlib 1.2.11 -5 24 MB/s 249 MB/s 630349312 180387584 28.62 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzfse 2017-03-08 41 MB/s 410 MB/s 630349312 180395667 28.62 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
ucl_nrv2e 1.03 -9 0.63 MB/s 245 MB/s 630349312 181304131 28.76 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
ucl_nrv2d 1.03 -9 0.62 MB/s 252 MB/s 630349312 183386226 29.09 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
yalz77 2015-09-19 -12 21 MB/s 282 MB/s 630349312 185068425 29.36 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
ucl_nrv2b 1.03 -9 0.64 MB/s 254 MB/s 630349312 186443368 29.58 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzo1b 2.09 -999 6.60 MB/s 560 MB/s 630349312 190437814 30.21 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lz4hc 1.8.0 -12 4.70 MB/s 1838 MB/s 630349312 195135857 30.96 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lizard 1.0 -19 3.84 MB/s 1960 MB/s 630349312 195334252 30.99 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
brotli24 2017-12-12 -1 135 MB/s 257 MB/s 630349312 196259442 31.14 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lz4hc 1.8.0 -10 11 MB/s 1822 MB/s 630349312 198009731 31.41 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzmat 1.01 15 MB/s 247 MB/s 630349312 198808150 31.54 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lizard 1.0 -39 3.85 MB/s 1847 MB/s 630349312 199874259 31.71 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
libdeflate 0.7 -1 110 MB/s 477 MB/s 630349312 200916411 31.87 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzg 1.0.8 -9 0.29 MB/s 545 MB/s 630349312 206970320 32.83 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
brieflz 1.1.0 97 MB/s 132 MB/s 630349312 221596487 35.15 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zlib 1.2.11 -1 63 MB/s 223 MB/s 630349312 224186115 35.57 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzvn 2017-03-08 33 MB/s 603 MB/s 630349312 228871658 36.31 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
quicklz 1.5.0 -3 37 MB/s 603 MB/s 630349312 232561738 36.89 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzrw 15-Jul-1991 -5 96 MB/s 406 MB/s 630349312 239712319 38.03 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
gipfeli 2016-07-13 195 MB/s 350 MB/s 630349312 240187191 38.10 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
yappy 2014-03-22 -99 54 MB/s 1582 MB/s 630349312 250628425 39.76 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
pithy 2011-12-24 -9 191 MB/s 1070 MB/s 630349312 254728440 40.41 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
density 0.12.5 beta -3 274 MB/s 243 MB/s 630349312 255431862 40.52 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lz4 1.8.0 296 MB/s 1698 MB/s 630349312 291940785 46.31 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lz4fast 1.8.0 -1 292 MB/s 1698 MB/s 630349312 291940785 46.31 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
slz_zlib 1.0.0 -3 136 MB/s 180 MB/s 630349312 296302604 47.01 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzf 3.6 -1 205 MB/s 393 MB/s 630349312 297405282 47.18 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
snappy 1.1.4 152 MB/s 748 MB/s 630349312 297988578 47.27 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
fastlz 0.1 -2 210 MB/s 339 MB/s 630349312 300420716 47.66 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
blosclz 2015-11-10 -9 170 MB/s 523 MB/s 630349312 305105501 48.40 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
wflz 2015-09-16 182 MB/s 483 MB/s 630349312 367481710 58.30 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzjb 2010 159 MB/s 291 MB/s 630349312 376145859 59.67 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lz4fast 1.8.0 -99 1210 MB/s 3384 MB/s 630349312 549307244 87.14 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
shrinker 0.1 178 MB/s 2471 MB/s 630349312 580903247 92.16 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
H:\smashshop_2018-Feb-13_Judaica>"turbobenchs_Official_v17.04_-_build_07_Apr_2017.exe" Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar -elibdeflate,12/oodle,19,49,89,112,114,116,118,129/lzsse2,17/lzturbo,19,12,29,22,39,32,49,59/zstd,12,22/lizard,19,29,39,49/brotli,11/lzma,9/bzip2/xpack,9/chameleon,2/density,3/lzham,4/trle/bsc,3,6/zpaq,2,5 -g -I1 -J31 -k1 -B2G
...
93978494 14.9 1.03 808.37 xpack 9 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
ERROR at 555847118:27, 6f
...
C Size ratio% C MB/s D MB/s Name File
65769435 10.4 0.51 160.51 lzturbo 49 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
68333375 10.8 0.32 0.32 zpaq 5 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
69936980 11.1 7.49 16.77 lzturbo 59 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
71556756 11.4 0.69 1092.81 lzturbo 39 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
73714983 11.7 1.31 134.87 lzma 9 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
74215638 11.8 0.88 303.07 lzham 4 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
74303904 11.8 0.31 1002.78 oodle 129 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
74304046 11.8 0.34 1001.22 oodle 89 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
76030170 12.1 0.94 693.12 zstd 22 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
76061255 12.1 0.20 679.63 oodle 19 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
87128650 13.8 0.73 1523.11 lzturbo 29 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
93978494 14.9 1.03 808.37 ?xpack 9 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
96487507 15.3 36.20 959.44 lzturbo 32 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
101468960 16.1 0.36 2522.43 oodle 118 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
102570170 16.3 0.33 2543.44 oodle 116 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
103339598 16.4 18.18 9.18 bsc 6 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
103619672 16.4 0.85 1721.37 lizard 49 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
104883928 16.6 0.86 1882.69 lizard 29 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
112593260 17.9 0.38 366.35 brotli 11 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
117557372 18.6 4.51 77.73 zpaq 2 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
120309827 19.1 9.72 25.28 bzip2 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
127030698 20.2 21.54 14.80 bsc 3 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
132294051 21.0 36.72 1354.83 lzturbo 22 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
138508624 22.0 7.67 574.53 zstd 12 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
142662359 22.6 22.33 2519.30 oodle 114 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
160509285 25.5 3.92 548.26 libdeflate 12 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
172562123 27.4 4.46 2572.05 lzsse2 17 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
195218661 31.0 0.91 2582.04 lzturbo 19 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
195334256 31.0 3.83 1968.91 lizard 19 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
195356086 31.0 2.81 1643.25 oodle 49 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
199874263 31.7 3.88 1853.17 lizard 39 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
215186827 34.1 41.28 2589.61 lzturbo 12 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
238259362 37.8 50.56 3119.61 oodle 112 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
255429524 40.5 229.04 245.98 density 3 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
348012680 55.2 1120.84 2265.59 chameleon 2 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
629643831 99.9 134.69 1448.73 trle Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
H:\smashshop_2018-Feb-13_Judaica>timer64 PPMd_varI_rev2_Intel15_32bit.exe e -o6 -m256 -fStephan_Kaze_http_unbound.biola.edu_103-bibles.tar.O6.PPMd_varI Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Fast PPMII compressor for textual data, variant I, Apr 3 2016
Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar:630349312 >111654445, 1.01 b
Kernel Time = 0.592 = 0%
User Time = 93.865 = 98%
Process Time = 94.458 = 99% Virtual Memory = 258 MB
Global Time = 95.109 = 100% Physical Memory = 259 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 PPMd_varI_rev2_Intel15_32bit.exe e -o16 -m256 -fStephan_Kaze_http_unbound.biola.edu_103-bibles.tar.O16.PPMd_varI Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Fast PPMII compressor for textual data, variant I, Apr 3 2016
Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar:630349312 >88490946, 1.05 bp
Kernel Time = 0.624 = 0%
User Time = 161.523 = 99%
Process Time = 162.147 = 99% Virtual Memory = 258 MB
Global Time = 163.148 = 100% Physical Memory = 259 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "7za_v16.04_x64.exe" a -tgzip -mx9 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MX9.zip Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
7-Zip (a) [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
Scanning the drive:
1 file, 630349312 bytes (602 MiB)
Creating archive: Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MX9.zip
Items to compress: 1
Files read from disk: 1
Archive size: 160721209 bytes (154 MiB)
Everything is Ok
Kernel Time = 0.374 = 0%
User Time = 876.990 = 99%
Process Time = 877.365 = 99% Virtual Memory = 6 MB
Global Time = 879.580 = 100% Physical Memory = 7 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "7za_v16.04_x64.exe" a -t7z -mx9 -md=29 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MX9Dict512.7z Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
7-Zip (a) [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
Scanning the drive:
1 file, 630349312 bytes (602 MiB)
Creating archive: Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MX9Dict512.7z
Items to compress: 1
Files read from disk: 1
Archive size: 72467053 bytes (70 MiB)
Everything is Ok
Kernel Time = 4.352 = 1%
User Time = 565.441 = 156%
Process Time = 569.793 = 157% Virtual Memory = 11299 MB
Global Time = 361.608 = 100% Physical Memory = 4710 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "xz_v5.2.3_x64.exe" -z -k -f -9 -e -v -v --lzma2=dict=512MiB --threads=1 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
xz_v5.2.3_x64: Filter chain: --lzma2=dict=512MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz_v5.2.3_x64: 5,378 MiB of memory is required. The limiter is disabled.
xz_v5.2.3_x64: Decompression will need 513 MiB of memory.
Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar (1/1)
100 % 67.3 MiB / 601.1 MiB = 0.112 1.2 MiB/s 8:06
Kernel Time = 3.276 = 0%
User Time = 479.032 = 98%
Process Time = 482.308 = 99% Virtual Memory = 5389 MB
Global Time = 487.063 = 100% Physical Memory = 4674 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "bsc_v3.1.0_x64.exe" e Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.ST6Block512.bsc -b512 -m6 -cp -Tt
This is bsc, Block Sorting Compressor. Version 3.1.0. 8 July 2012.
Copyright (c) 2009-2012 Ilya Grebnov <Ilya.Grebnov@gmail.com>.
Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar compressed 630349312 into 100258206 in 38.189 seconds.
Kernel Time = 1.326 = 3%
User Time = 34.367 = 86%
Process Time = 35.693 = 90% Virtual Memory = 2604 MB
Global Time = 39.632 = 100% Physical Memory = 2599 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "zpaq_v7.05_x64.exe" add Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.method29.zpaq Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar -method 29 -threads 1
zpaq v7.05 journaling archiver, compiled Apr 17 2015
Adding 630.349312 MB in 1 files -method 29 -threads 1 at 2017-12-02 13:28:35.
85.30% 0:00:01 [1..7696] 536743069 -method 29,209,1
100.00% 0:00:00 + Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar 630349312 -> 629301380
100.00% 0:00:00 [7697..9011] 92594371 -method 29,211,1
1 +added, 0 -removed.
0.000000 + (630.349312 -> 629.301380 -> 93.022200) = 93.022200 MB
167.467 seconds (all OK)
Kernel Time = 1.232 = 0%
User Time = 166.172 = 99%
Process Time = 167.404 = 99% Virtual Memory = 3531 MB
Global Time = 167.521 = 100% Physical Memory = 3044 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "zpaq_v7.05_x64.exe" add Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.method59.zpaq Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar -method 59 -threads 1
zpaq v7.05 journaling archiver, compiled Apr 17 2015
Adding 630.349312 MB in 1 files -method 59 -threads 1 at 2017-12-02 13:31:22.
85.30% 0:00:01 [1..7696] 536743069 -method 59,209,1
100.00% 0:00:00 + Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar 630349312 -> 629301380
100.00% 0:00:00 [7697..9011] 92594371 -method 59,211,1
1 +added, 0 -removed.
0.000000 + (630.349312 -> 629.301380 -> 54.815778) = 54.815778 MB
2414.443 seconds (all OK)
Kernel Time = 4.820 = 0%
User Time = 2408.951 = 99%
Process Time = 2413.772 = 99% Virtual Memory = 6473 MB
Global Time = 2414.448 = 100% Physical Memory = 4707 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 lz5 -49 -B7 --no-frame-crc Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
using blocks of size 262144 KB
Compressed filename will be : Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.lz5
Compressed 630349312 bytes into 90740613 bytes ==> 14.40%
Kernel Time = 0.624 = 0%
User Time = 727.432 = 99%
Process Time = 728.056 = 99% Virtual Memory = 676 MB
Global Time = 731.968 = 100% Physical Memory = 459 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "nanozip-0.09a.win64.exe" a -t1 -cc Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.cc.nz Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
NanoZip 0.09 alpha/Win64 (C) 2008-2011 Sami Runsas www.nanozip.net
Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz|32939 MHz|#2+HT|15758/16332 MB
Archive: Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.cc.nz
Threads: 1, memory: 512 MB
Compressor #0: nz_cm [524 MB]
Compressed 630 349 312 into 60 933 816 in 15m 56.48s, 644 KB/s
IO-in: 0.37s, 1612 MB/s. IO-out: 0.02s, 2075 MB/s
Kernel Time = 1.419 = 0%
User Time = 977.517 = 99%
Process Time = 978.937 = 99% Virtual Memory = 539 MB
Global Time = 980.324 = 100% Physical Memory = 511 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "nanozip-0.09a.win64.exe" a -t1 -co Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.co.nz Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
NanoZip 0.09 alpha/Win64 (C) 2008-2011 Sami Runsas www.nanozip.net
Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz|29616 MHz|#2+HT|15753/16332 MB
Archive: Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.co.nz
Threads: 1, memory: 512 MB
Compressor #0: nz_optimum1 [506 MB]
Compressed 630 349 312 into 71 362 322 in 2m 15.55s, 4541 KB/s
IO-in: 0.37s, 1625 MB/s. IO-out: 0.02s, 2959 MB/s
Kernel Time = 14.055 = 10%
User Time = 124.941 = 89%
Process Time = 138.996 = 99% Virtual Memory = 524 MB
Global Time = 139.514 = 100% Physical Memory = 516 MB
CABARC, Microsoft (R) Cabinet Tool - Version 5.1.2600.0, Copyright (c) Microsoft Corporation.
H:\smashshop_2018-Feb-13_Judaica>timer64 cabarc.exe -m LZX:21 N Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.LZX21.cab Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Microsoft (R) Cabinet Tool - Version 5.1.2600.0
Copyright (c) Microsoft Corporation. All rights reserved..
Creating new cabinet 'Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.LZX21.cab' with compression 'LZX:21':
-- adding Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Completed successfully
Kernel Time = 1.263 = 0%
User Time = 628.028 = 98%
Process Time = 629.292 = 99% Virtual Memory = 20 MB
Global Time = 634.587 = 100% Physical Memory = 21 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 cabarc.exe -m MSZIP N Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MSZIP.cab Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Microsoft (R) Cabinet Tool - Version 5.1.2600.0
Copyright (c) Microsoft Corporation. All rights reserved..
Creating new cabinet 'Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MSZIP.cab' with compression 'MSZIP':
-- adding Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Completed successfully
Kernel Time = 0.826 = 0%
User Time = 145.346 = 95%
Process Time = 146.172 = 95% Virtual Memory = 1 MB
Global Time = 152.982 = 100% Physical Memory = 3 MB
Compress, version: (N)compress 4.2.4.4, compiled: Fri, Aug 23, 2013 11:56:09. Authors: Peter Jannesen, Dave Mack, Spencer W. Thomas, Jim McKie, Steve Davies, Ken Turkowski, James A. Woods, Joe Orost.
H:\smashshop_2018-Feb-13_Judaica>timer64 compress.exe -c Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar 1>Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.Z
H:\smashshop_2018-Feb-13_Judaica>timer64 zstd-v1.3.3-win64.exe --ultra -22 --zstd=wlog=29,clog=30,hlog=30,slog=26 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar : 11.51% (630349312 => 72525795 bytes, Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.zst)
Kernel Time = 3.728 = 0%
User Time = 715.794 = 97%
Process Time = 719.523 = 98% Virtual Memory = 8724 MB
Global Time = 730.412 = 100% Physical Memory = 8330 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 rz_1.01.exe a -d 512M Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.512M.rz Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
*** RAZOR Archiver 1.01 (2017-09-14) - DEMO/TEST version ***
*** (c) Christian Martelock (christian.martelock@web.de) ***
Scanning h:\smashshop_2018-feb-13_judaica\stephan_kaze_http_unbound.biola.edu_103-bibles.tar
Found 0 dirs, 1 files, 630349312 bytes.
Creating archive Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.512M.rz
Window : 524288K (4096M..1024G)
Header : 82
Size : 58022323
Archive ok. Added 0 dirs, 1 files, 630349312 bytes.
CPU time = 2364.304s / wall time = 1871.536s
Kernel Time = 11.372 = 0%
User Time = 2364.304 = 126%
Process Time = 2375.676 = 126% Virtual Memory = 6486 MB
Global Time = 1871.602 = 100% Physical Memory = 5458 MB
54,815,778 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.method59.zpaq
58,022,323 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.512M.rz
60,933,816 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.cc.nz
70,592,460 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.L9Dict512.xz
71,362,322 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.co.nz
72,467,053 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MX9Dict512.7z
72,525,795 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.zst
88,491,012 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.O16.PPMd_varI
90,740,613 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.M49Block256.lz5
93,022,200 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.method29.zpaq
100,258,206 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.ST6Block512.bsc
111,654,511 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.O6.PPMd_varI
122,224,313 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.LZX21.cab
160,721,209 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MX9.zip
172,450,102 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.MSZIP.cab
224,907,813 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.Z
630,349,312 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
H:\smashshop_2018-Feb-13_Judaica>timer64 lz4_v1_8_1_win64 -12 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.12.lz4
Compressed 630349312 bytes into 195589287 bytes ==> 31.03%
Kernel Time = 0.265 = 0%
User Time = 139.137 = 98%
Process Time = 139.402 = 98% Virtual Memory = 10 MB
Global Time = 141.151 = 100% Physical Memory = 8 MB
One highlighted fragment:
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
...
lzma 16.04 -9 1.31 MB/s 134 MB/s 630349312 73714979 11.69 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
brotli24 2017-12-12 -11 0.42 MB/s 506 MB/s 630349312 74375172 11.80 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
csc 2016-10-13 -5 2.10 MB/s 105 MB/s 630349312 76509932 12.14 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zstd24 1.3.3 -22 1.05 MB/s 701 MB/s 630349312 77176177 12.24 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
tornado 0.6a -16 1.11 MB/s 291 MB/s 630349312 77524331 12.30 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzham 1.0 -d26 -4 0.88 MB/s 287 MB/s 630349312 77755762 12.34 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
lzham24 1.0 -4 0.92 MB/s 286 MB/s 630349312 78373249 12.43 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
zstd24LDM 1.3.3 -22 1.63 MB/s 702 MB/s 630349312 79586664 12.63 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
...
C Size ratio% C MB/s D MB/s Name File
...
71556756 11.4 0.69 1092.81 lzturbo 39 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
73714983 11.7 1.31 134.87 lzma 9 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
74215638 11.8 0.88 303.07 lzham 4 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
74303904 11.8 0.31 1002.78 oodle 129 Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar.tbb
...
Brotli and Zstd with their 16MB windows are so strong and fast, however, Oodle 'Hydra' is amazing, LzTurbo 39 even more so.
Entropy of Alphabetical Orders Table, in short, Decomposition Table, or, B.B.s Table:
--------------------------------------------------------------------------------------------------------------------
| #1 Filesize | #2 | #3 Granularity | #4 N.U.B.B.s | #5 U.B.B.s | #6 U.B.B.s_Alphabet | #7 Entropy | #8 Filename
| 000,630,349,312 | 001 | 008bit | 000,630,349,312 | 000,000,000,198 | NewOrder = 008bit | 0.000,000,314 | Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
| 000,630,349,312 | 002 | 016bit | 000,630,349,311 | 000,000,014,354 | NewOrder = 014bit | 0.000,022,771 | Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
| 000,630,349,312 | 004 | 032bit | 000,630,349,309 | 000,000,988,802 | NewOrder = 020bit | 0.001,568,657 | Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
| 000,630,349,312 | 008 | 064bit | 000,630,349,305 | 000,044,530,095 | NewOrder = 026bit | 0.070,643,520 | Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
| 000,630,349,312 | 064 | 512bit | 000,630,349,249 | 000,427,775,627 | NewOrder = 029bit | 0.678,632,704 | Stephan_Kaze_http_unbound.biola.edu_103-bibles.tar
| 001,382,122,496 | 001 | 008bit | 001,382,122,496 | 000,000,000,256 | NewOrder = 008bit | 0.000,000,185 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 002 | 016bit | 001,382,122,495 | 000,000,065,536 | NewOrder = 016bit | 0.000,047,416 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 004 | 032bit | 001,382,122,493 | 000,018,815,000 | NewOrder = 025bit | 0.013,613,121 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 008 | 064bit | 001,382,122,489 | 000,142,060,792 | NewOrder = 028bit | 0.102,784,520 | TEXTFILES.COM_(58096_files).tar
| 001,382,122,496 | 064 | 512bit | 001,382,122,433 | 001,115,907,301 | NewOrder = 031bit | 0.807,386,688 | TEXTFILES.COM_(58096_files).tar
| 002,037,880,832 | 001 | 008bit | 002,037,880,832 | 000,000,000,222 | NewOrder = 008bit | 0.000,000,108 | INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
| 002,037,880,832 | 002 | 016bit | 002,037,880,831 | 000,000,010,866 | NewOrder = 014bit | 0.000,005,332 | INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
| 002,037,880,832 | 004 | 032bit | 002,037,880,829 | 000,001,318,754 | NewOrder = 021bit | 0.000,647,120 | INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
| 002,037,880,832 | 008 | 064bit | 002,037,880,825 | 000,084,505,728 | NewOrder = 027bit | 0.041,467,456 | INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
| 002,037,880,832 | 064 | 512bit | 002,037,880,769 | 001,682,274,615 | NewOrder = 031bit | 0.825,501,888 | INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
--------------------------------------------------------------------------------------------------------------------
Table Legend:
#1: Filesize (in bytes)
#2: Building-Blocks_Order = B.B.s_Order (in bytes)
#3: Granularity = Alphabet_Order = "Letters"_size = B.B.s_Order x 8bit
#4: N.U.B.B.s = Non-Unique-Building-Blocks = (Filesize - B.B.s_Order + 1)
#5: U.B.B.s = Unique-Building-Blocks
#6: Shrunk_Alphabet (to house U.B.B.s) = NewOrder, for (NewOrder=1; UBBs > ((1LL)<<NewOrder); NewOrder++);
#7: Entropy_Alphabetical = U.B.B.s / N.U.B.B.s; LOWER -> MORE COMPRESSIBLE; The bigger the number the higher the "diffusion" i.e.
richness of combinations, when 1 then full coverage of Entropy spectrum is present, lower values suggest "textuality",
higher Alphabetical Orders could "warn" for duplicative fragments
#8: Filename
And the ISTA:
H:\smashshop_2018-Feb-13_Judaica>lzbench173 -c4 -i1,15 -o3 -ebrotli24,1,5,11/tornado,16/blosclz,9/brieflz/crush,2/csc,5/density,3/fastlz,2/gipfeli/zstd24,12,22/zstd24LDM,12,22/lzo1b,999/lzham,4/lzham24,4/libdeflate,1,12/lz4fast,1,99/lz4/lz4hc,10,12/lizard,19,29,39,49/lzf,1/lzfse/lzg,9/lzham,1/lzjb/lzlib,9/lzma,9/lzrw,5/lzsse2,17/lzsse4,17/lzsse8,17/lzvn/pithy,9/quicklz,3/snappy/slz_zlib,3/ucl_nrv2b,9/ucl_nrv2d,9/ucl_nrv2e,9/xpack,9/xz,9/yalz77,12/yappy,99/zlib,1,5,9/zling,4/shrinker/wflz/lzmat "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
lzbench 1.7.3 (64-bit Windows) Assembled by P.Skibinski
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
memcpy 4980 MB/s 6266 MB/s 2037880832 2037880832 100.00 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
...
csc 2016-10-13 -5 2.07 MB/s ERROR 2037880832 340223046 16.69 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
...
The results sorted by column number 4:
Compressor name Compress. Decompress. Orig. size Compr. size Ratio Filename
xz 5.2.3 -9 1.16 MB/s 89 MB/s 2037880832 333183152 16.35 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzma 16.04 -9 1.08 MB/s 98 MB/s 2037880832 333185669 16.35 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzlib 1.8 -9 0.85 MB/s 63 MB/s 2037880832 334244294 16.40 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzham 1.0 -d26 -4 0.72 MB/s 200 MB/s 2037880832 338656530 16.62 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
csc 2016-10-13 -5 2.07 MB/s ERROR 2037880832 340223046 16.69 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
tornado 0.6a -16 0.96 MB/s 194 MB/s 2037880832 340988334 16.73 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
brotli24 2017-12-12 -11 0.36 MB/s 312 MB/s 2037880832 344308436 16.90 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzham24 1.0 -4 0.83 MB/s 203 MB/s 2037880832 349705856 17.16 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zstd24 1.3.3 -22 1.00 MB/s 504 MB/s 2037880832 350072648 17.18 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zstd24LDM 1.3.3 -22 1.24 MB/s 505 MB/s 2037880832 364809558 17.90 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zling 2016-01-10 -4 33 MB/s 178 MB/s 2037880832 407797091 20.01 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzham 1.0 -d26 -1 1.91 MB/s 188 MB/s 2037880832 409579714 20.10 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zstd24LDM 1.3.3 -12 8.27 MB/s 543 MB/s 2037880832 424043476 20.81 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zstd24 1.3.3 -12 8.57 MB/s 538 MB/s 2037880832 424231365 20.82 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
crush 1.0 -2 1.65 MB/s 325 MB/s 2037880832 436392632 21.41 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
brotli24 2017-12-12 -5 20 MB/s 360 MB/s 2037880832 437980071 21.49 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
xpack 2016-06-02 -9 10 MB/s 621 MB/s 2037880832 464249082 22.78 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lizard 1.0 -49 0.57 MB/s 1093 MB/s 2037880832 475168369 23.32 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lizard 1.0 -29 0.57 MB/s 1244 MB/s 2037880832 482552346 23.68 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
libdeflate 0.7 -12 4.15 MB/s 575 MB/s 2037880832 488688423 23.98 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zlib 1.2.11 -9 9.90 MB/s 274 MB/s 2037880832 514394996 25.24 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzfse 2017-03-08 43 MB/s 484 MB/s 2037880832 517205091 25.38 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzsse2 2016-05-14 -17 1.01 MB/s 2604 MB/s 2037880832 521548673 25.59 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzsse4 2016-05-14 -17 1.04 MB/s 2620 MB/s 2037880832 525368252 25.78 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
ucl_nrv2e 1.03 -9 1.06 MB/s 266 MB/s 2037880832 528302475 25.92 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zlib 1.2.11 -5 30 MB/s 267 MB/s 2037880832 532930185 26.15 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzsse8 2016-05-14 -17 1.02 MB/s 2474 MB/s 2037880832 533668053 26.19 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
ucl_nrv2d 1.03 -9 1.07 MB/s 276 MB/s 2037880832 533919837 26.20 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
ucl_nrv2b 1.03 -9 1.10 MB/s 275 MB/s 2037880832 544922667 26.74 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
brotli24 2017-12-12 -1 147 MB/s 286 MB/s 2037880832 557707702 27.37 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzo1b 2.09 -999 10 MB/s 599 MB/s 2037880832 564565204 27.70 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lz4hc 1.8.0 -12 5.11 MB/s 1992 MB/s 2037880832 575897244 28.26 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lizard 1.0 -19 1.75 MB/s 2204 MB/s 2037880832 576729202 28.30 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lz4hc 1.8.0 -10 15 MB/s 1968 MB/s 2037880832 584647498 28.69 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzmat 1.01 21 MB/s 262 MB/s 2037880832 588123531 28.86 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lizard 1.0 -39 1.74 MB/s 2065 MB/s 2037880832 588867589 28.90 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
libdeflate 0.7 -1 115 MB/s 512 MB/s 2037880832 590079525 28.96 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzg 1.0.8 -9 0.51 MB/s 565 MB/s 2037880832 601868196 29.53 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
yalz77 2015-09-19 -12 18 MB/s 248 MB/s 2037880832 638511955 31.33 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
zlib 1.2.11 -1 71 MB/s 246 MB/s 2037880832 641418204 31.47 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
brieflz 1.1.0 102 MB/s 142 MB/s 2037880832 643812920 31.59 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
quicklz 1.5.0 -3 39 MB/s 706 MB/s 2037880832 657657202 32.27 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzvn 2017-03-08 35 MB/s 704 MB/s 2037880832 660682678 32.42 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
gipfeli 2016-07-13 211 MB/s 362 MB/s 2037880832 693739855 34.04 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
density 0.12.5 beta -3 234 MB/s 239 MB/s 2037880832 694222204 34.07 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzrw 15-Jul-1991 -5 104 MB/s 424 MB/s 2037880832 718189499 35.24 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
pithy 2011-12-24 -9 228 MB/s 1169 MB/s 2037880832 719981430 35.33 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
slz_zlib 1.0.0 -3 154 MB/s 199 MB/s 2037880832 830975176 40.78 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
yappy 2014-03-22 -99 56 MB/s 1644 MB/s 2037880832 831056776 40.78 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
fastlz 0.1 -2 229 MB/s 387 MB/s 2037880832 833368341 40.89 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lz4fast 1.8.0 -1 329 MB/s 1912 MB/s 2037880832 850293388 41.72 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lz4 1.8.0 334 MB/s 1915 MB/s 2037880832 850293388 41.72 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
blosclz 2015-11-10 -9 184 MB/s 628 MB/s 2037880832 860245709 42.21 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzf 3.6 -1 222 MB/s 427 MB/s 2037880832 860781071 42.24 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
snappy 1.1.4 174 MB/s 845 MB/s 2037880832 868293407 42.61 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
wflz 2015-09-16 201 MB/s 582 MB/s 2037880832 997147394 48.93 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lzjb 2010 175 MB/s 327 MB/s 2037880832 1181870649 58.00 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
lz4fast 1.8.0 -99 1443 MB/s 3805 MB/s 2037880832 1632484887 80.11 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
shrinker 0.1 115 MB/s 5666 MB/s 2037880832 2023752814 99.31 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
H:\smashshop_2018-Feb-13_Judaica>"turbobenchs_Official_v17.04_-_build_07_Apr_2017.exe" "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar" -elibdeflate,12/oodle,19,49,89,112,114,116,118,129/lzsse2,17/lzturbo,19,12,29,22,39,32,49,59/zstd,12,22/lizard,19,29,39,49/brotli,11/lzma,9/bzip2/xpack,9/chameleon,2/density,3/lzham,4/trle/bsc,3,6/zpaq,2,5 -g -I1 -J31 -k1 -B2G
...
VirtualAlloc failed lzturbo 29
...
VirtualAlloc failed lzturbo 49
...
ERROR at 539935432:68, 74
ERROR at 544902976:44, 6f
416961699 20.5 0.65 761.34 ?xpack 9 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
...
C Size ratio% C MB/s D MB/s Name File
260338832 12.8 0.30 0.30 zpaq 5 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
273237972 13.4 6.20 14.20 lzturbo 59 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
291401772 14.3 18.19 7.86 bsc 6 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
324042669 15.9 0.55 498.78 lzturbo 39 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
328703215 16.1 0.66 198.06 lzham 4 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
333215641 16.4 1.06 98.76 lzma 9 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
336460596 16.5 0.76 483.39 zstd 22 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
341176401 16.7 0.25 504.27 oodle 89 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
341185735 16.7 0.23 505.42 oodle 129 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
350504364 17.2 0.16 366.37 oodle 19 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
354038918 17.4 9.23 26.80 bzip2 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
357203236 17.5 0.37 361.44 brotli 11 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
359485284 17.6 23.65 15.24 bsc 3 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
416961699 20.5 0.65 761.34 ?xpack 9 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
419171865 20.6 26.35 563.13 lzturbo 32 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
424236354 20.8 8.40 611.39 zstd 12 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
435380248 21.4 3.95 69.74 zpaq 2 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
475213911 23.3 0.57 1094.42 lizard 49 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
482598265 23.7 0.57 1247.65 lizard 29 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
488687894 24.0 4.17 542.31 libdeflate 12 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
495266014 24.3 0.24 1845.62 oodle 116 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
500268429 24.5 0.26 1834.90 oodle 118 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
521547896 25.6 0.87 2607.54 lzsse2 17 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
568510761 27.9 26.07 945.30 lzturbo 22 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
576725033 28.3 3.02 1904.77 oodle 49 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
576726425 28.3 1.77 2587.60 lizard 19 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
576751789 28.3 0.66 2916.07 lzturbo 19 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
588826444 28.9 1.77 2071.96 lizard 39 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
596321123 29.3 14.46 2191.59 oodle 114 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
633867923 31.1 44.85 3350.90 lzturbo 12 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
694234382 34.1 266.36 247.94 density 3 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
698032694 34.3 50.56 2684.29 oodle 112 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
1135395872 55.7 1113.69 2519.89 chameleon 2 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
1944549874 95.4 147.69 1489.27 trle INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
H:\smashshop_2018-Feb-13_Judaica>timer64 PPMd_varI_rev2_Intel15_32bit.exe e -o6 -m256 -f"INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".O6.PPMd_varI "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
Fast PPMII compressor for textual data, variant I, Apr 3 2016
INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar:203788083
Kernel Time = 1.591 = 0%
User Time = 279.428 = 98%
Process Time = 281.020 = 99% Virtual Memory = 258 MB
Global Time = 282.283 = 100% Physical Memory = 259 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 PPMd_varI_rev2_Intel15_32bit.exe e -o16 -m256 -f"INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".O16.PPMd_varI "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
Fast PPMII compressor for textual data, variant I, Apr 3 2016
INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar:203788083
Kernel Time = 1.575 = 0%
User Time = 503.430 = 99%
Process Time = 505.006 = 99% Virtual Memory = 258 MB
Global Time = 507.741 = 100% Physical Memory = 259 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "7za_v16.04_x64.exe" a -tgzip -mx9 "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".MX9.zip "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
7-Zip (a) [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
Scanning the drive:
1 file, 2037880832 bytes (1944 MiB)
Creating archive: INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.MX9.zip
Items to compress: 1
Files read from disk: 1
Archive size: 488604001 bytes (466 MiB)
Everything is Ok
Kernel Time = 0.951 = 0%
User Time = 2547.808 = 99%
Process Time = 2548.759 = 99% Virtual Memory = 6 MB
Global Time = 2550.813 = 100% Physical Memory = 7 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "7za_v16.04_x64.exe" a -t7z -mx9 -md=29 "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".MX9Dict512.7z "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
7-Zip (a) [64] 16.04 : Copyright (c) 1999-2016 Igor Pavlov : 2016-10-04
Scanning the drive:
1 file, 2037880832 bytes (1944 MiB)
Creating archive: INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.MX9Dict512.7z
Items to compress: 1
Files read from disk: 1
Archive size: 326955812 bytes (312 MiB)
Everything is Ok
Kernel Time = 14.664 = 1%
User Time = 2772.012 = 270%
Process Time = 2786.677 = 271% Virtual Memory = 11299 MB
Global Time = 1025.780 = 100% Physical Memory = 9419 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "xz_v5.2.3_x64.exe" -z -k -f -9 -e -v -v --lzma2=dict=512MiB --threads=1 "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
xz_v5.2.3_x64: Filter chain: --lzma2=dict=512MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz_v5.2.3_x64: 5,378 MiB of memory is required. The limiter is disabled.
xz_v5.2.3_x64: Decompression will need 513 MiB of memory.
INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar (1/1)
100 % 307.3 MiB / 1,943.5 MiB = 0.158 895 KiB/s 37:03
Kernel Time = 9.360 = 0%
User Time = 2188.787 = 98%
Process Time = 2198.147 = 98% Virtual Memory = 5389 MB
Global Time = 2224.158 = 100% Physical Memory = 4991 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "bsc_v3.1.0_x64.exe" e "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar" "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".ST6Block512.bsc -b512 -m6 -cp -Tt
This is bsc, Block Sorting Compressor. Version 3.1.0. 8 July 2012.
Copyright (c) 2009-2012 Ilya Grebnov <Ilya.Grebnov@gmail.com>.
INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_ compressed 2037880832 into 286458738 in 121.229 seconds.
Kernel Time = 3.931 = 3%
User Time = 109.294 = 89%
Process Time = 113.225 = 92% Virtual Memory = 2497 MB
Global Time = 122.292 = 100% Physical Memory = 2492 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "zpaq_v7.05_x64.exe" add "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".method29.zpaq "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar" -method 29 -threads 1
zpaq v7.05 journaling archiver, compiled Apr 17 2015
Adding 2037.880832 MB in 1 files -method 29 -threads 1 at 2017-12-03 23:12:25.
26.35% 0:00:19 [1..6825] 536820051 -method 29,202,1
52.69% 0:00:13 [6826..15672] 536802912 -method 29,205,1
79.03% 0:00:50 [15673..22943] 536678139 -method 29,198,1
100.00% 0:00:00 + INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar 2037880832 -> 2037699891
100.00% 0:00:00 [22944..28815] 427514081 -method 29,195,1
1 +added, 0 -removed.
0.000000 + (2037.880832 -> 2037.699891 -> 402.995397) = 402.995397 MB
719.742 seconds (all OK)
Kernel Time = 3.650 = 0%
User Time = 732.377 = 101%
Process Time = 736.028 = 102% Virtual Memory = 3532 MB
Global Time = 720.187 = 100% Physical Memory = 3469 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "zpaq_v7.05_x64.exe" add "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".method59.zpaq "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar" -method 59 -threads 1
zpaq v7.05 journaling archiver, compiled Apr 17 2015
Adding 2037.880832 MB in 1 files -method 59 -threads 1 at 2017-12-03 23:24:24.
26.35% 0:00:19 [1..6825] 536820051 -method 59,202,1
52.69% 0:00:13 [6826..15672] 536802912 -method 59,205,1
79.03% 0:09:09 [15673..22943] 536678139 -method 59,198,1
100.00% 0:00:00 + INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar 2037880832 -> 2037699891
100.00% 0:00:00 [22944..28815] 427514081 -method 59,195,1
1 +added, 0 -removed.
0.000000 + (2037.880832 -> 2037.699891 -> 242.738948) = 242.738948 MB
8032.694 seconds (all OK)
Kernel Time = 12.511 = 0%
User Time = 8026.064 = 99%
Process Time = 8038.575 = 100% Virtual Memory = 6473 MB
Global Time = 8032.713 = 100% Physical Memory = 5286 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 lz5 -49 -B7 --no-frame-crc "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
using blocks of size 262144 KB
Compressed filename will be : INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.lz5
Compressed 2037880832 bytes into 410539971 bytes ==> 20.15%
Kernel Time = 2.137 = 0%
User Time = 3470.538 = 99%
Process Time = 3472.675 = 99% Virtual Memory = 676 MB
Global Time = 3488.992 = 100% Physical Memory = 482 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "nanozip-0.09a.win64.exe" a -t1 -cc "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".cc.nz "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
NanoZip 0.09 alpha/Win64 (C) 2008-2011 Sami Runsas www.nanozip.net
Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz|47377 MHz|#2+HT|15673/16332 MB
Archive: INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.cc.nz
Threads: 1, memory: 512 MB
Compressor #0: nz_cm [524 MB]
Compressed 2 037 880 832 into 242 424 164 in 41m 42.41s, 795 KB/s
IO-in: 1.15s, 1687 MB/s. IO-out: 0.17s, 1306 MB/s
Kernel Time = 5.600 = 0%
User Time = 2556.013 = 99%
Process Time = 2561.614 = 99% Virtual Memory = 539 MB
Global Time = 2564.954 = 100% Physical Memory = 514 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 "nanozip-0.09a.win64.exe" a -t1 -co "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".co.nz "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
NanoZip 0.09 alpha/Win64 (C) 2008-2011 Sami Runsas www.nanozip.net
Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz|3351 MHz|#2+HT|15662/16332 MB
Archive: INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.co.nz
Threads: 1, memory: 512 MB
Compressor #0: nz_optimum1 [506 MB]
Compressed 2 037 880 832 into 271 454 594 in 5m 13.77s, 6342 KB/s
IO-in: 1.17s, 1650 MB/s. IO-out: 0.15s, 1638 MB/s
Kernel Time = 13.228 = 4%
User Time = 309.100 = 95%
Process Time = 322.329 = 99% Virtual Memory = 527 MB
Global Time = 323.384 = 100% Physical Memory = 516 MB
CABARC, Microsoft (R) Cabinet Tool - Version 5.1.2600.0, Copyright (c) Microsoft Corporation.
H:\smashshop_2018-Feb-13_Judaica>timer64 cabarc.exe -m LZX:21 N "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".LZX21.cab "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
Microsoft (R) Cabinet Tool - Version 5.1.2600.0
Copyright (c) Microsoft Corporation. All rights reserved..
Creating new cabinet 'INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.LZX21.cab' with compression 'LZX:21':
-- adding INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
Completed successfully
Kernel Time = 3.338 = 0%
User Time = 1582.505 = 98%
Process Time = 1585.843 = 99% Virtual Memory = 20 MB
Global Time = 1600.515 = 100% Physical Memory = 21 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 cabarc.exe -m MSZIP N "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".MSZIP.cab "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
Microsoft (R) Cabinet Tool - Version 5.1.2600.0
Copyright (c) Microsoft Corporation. All rights reserved..
Creating new cabinet 'INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.MSZIP.cab' with compression 'MSZIP':
-- adding INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
Completed successfully
Kernel Time = 3.073 = 0%
User Time = 288.196 = 92%
Process Time = 291.269 = 93% Virtual Memory = 1 MB
Global Time = 310.206 = 100% Physical Memory = 3 MB
Compress, version: (N)compress 4.2.4.4, compiled: Fri, Aug 23, 2013 11:56:09. Authors: Peter Jannesen, Dave Mack, Spencer W. Thomas, Jim McKie, Steve Davies, Ken Turkowski, James A. Woods, Joe Orost.
H:\smashshop_2018-Feb-13_Judaica>timer64 compress.exe -c "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar" 1>"INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".Z
H:\smashshop_2018-Feb-13_Judaica>timer64 zstd-v1.3.3-win64.exe --ultra -22 --zstd=wlog=29,clog=30,hlog=30,slog=26 "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar : 16.10% (2037880832 => 328025870 bytes, INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.zst)
Kernel Time = 12.292 = 0%
User Time = 5364.437 = 99%
Process Time = 5376.730 = 99% Virtual Memory = 8724 MB
Global Time = 5418.360 = 100% Physical Memory = 8324 MB
H:\smashshop_2018-Feb-13_Judaica>timer64 rz_1.01.exe a -d 512M "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".512M.rz "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar"
*** RAZOR Archiver 1.01 (2017-09-14) - DEMO/TEST version ***
*** (c) Christian Martelock (christian.martelock@web.de) ***
Scanning h:\smashshop_2018-feb-13_judaica\internet_sacred_text_archive_dvd-rom_9_(english_140479_htm_files).tar
Found 0 dirs, 1 files, 2037880832 bytes.
Creating archive INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.512M.rz
Window : 524288K (4096M..1024G)
Header : 98
Size : 273746635
Archive ok. Added 0 dirs, 1 files, 2037880832 bytes.
CPU time = 10106.057s / wall time = 8022.831s
Kernel Time = 37.362 = 0%
User Time = 10106.056 = 125%
Process Time = 10143.419 = 126% Virtual Memory = 6500 MB
Global Time = 8023.054 = 100% Physical Memory = 6063 MB
242,424,164 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.cc.nz
242,738,948 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.method59.zpaq
271,454,594 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.co.nz
273,746,635 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.512M.rz
281,540,748 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.O16.PPMd_varI
286,458,738 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.ST6Block512.bsc
303,601,274 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.O6.PPMd_varI
322,263,828 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.L9Dict512.xz
326,955,812 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.MX9Dict512.7z
328,025,870 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.zst
375,626,108 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.LZX21.cab
402,995,397 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.method29.zpaq
410,539,971 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.M49Block256.lz5
488,604,001 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.MX9.zip
516,953,204 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.MSZIP.cab
698,388,413 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar.Z
2,037,880,832 INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar
H:\smashshop_2018-Feb-13_Judaica>timer64 lz4_v1_8_1_win64 -12 "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar" "INTERNET_SACRED_TEXT_ARCHIVE_DVD-ROM_9_(English_140479_htm_files).tar".12.lz4
Compressed 2037880832 bytes into 577475691 bytes ==> 28.34%
Kernel Time = 0.920 = 0%
User Time = 411.468 = 99%
Process Time = 412.388 = 99% Virtual Memory = 10 MB
Global Time = 415.403 = 100% Physical Memory = 8 MB
Not happy with the test, RAM is insufficient for my favorite mode - LzTurbo 29 - also this old 2nd gen Intel paired with such slow RAM is not showing the modern picture, kinda retroism instead of modernism. Or, like the F1 bolids forced to race in streets of Monte Carlo - never got the idea, crash at Monaco:
Next one will be the 900MB DNA sequence of the Dragonfly... Hopefully, at the end of this week, BROTLI with 1GB window will be released, will wait until it comes, interesting to see it brotlifying (in all modes) the genome of a species 350,000,000 years old. The showdown will be intensified by Oodle 'Hydra' and LzTurbo 39.
Thanks a lot for the nifty TurboBench, after wandering here and there, finally thought it is a good idea to post my endless quest (in form of console logs and tables) for giving a rich picture of [de]compressors of today. Hope, you don't mind me posting on a weekly basis inhere. My wish is we to have on a public site (GitHub is fully okay) as many as possible interesting datasets tested.
The older package used for tests:
The single PDF file unfinished (134 pages) overview of Textual Benchmarks (done on my other laptop with i5-7200u) is here: https://drive.google.com/file/d/1DUVowDtp__WGiC3HIw1whE2dTKt7kvBD/view?usp=sharing
Wanted to be much more versatile and bigger, but it is what it is.
The newer package used for tests:
The testmachine will be (already acquired) the Toshiba laptop that I used to torture:
My wish was (has to wait) to buy a dedicated machine for heavy compression benchmarks with i3-8100 and 64GB DDR4: http://www.overclock.net/forum/158-laptops-netbooks/1619592-kaby-lake-intel-core-i5-7200u-how-fast.html#post26536756 The key thing inhere is the eventual running of any file (<1.5GB due to free RAM being ~14GB). It would be nice once and for all we to have one versatile roster. My preference towards textual data remains, so the files listed below will be run first.
Above dump is only a candidate, I am open for suggestions as to what additional compressors/options to be added. The package used is this: smashshop_2018-Feb-13_Judaica.zip 226 MB (237,723,608 bytes)