search

jibsen / brieflz

Small fast Lempel-Ziv compression library
zlib License
101 stars 18 forks source link

Binary needed, ICL compile line needed, standalone unpacker needed #7

Closed Sanmayce closed 6 years ago

Sanmayce commented 6 years ago

Hi Jørgen, seeing your new stronger modes gladdened my eyes, your impressive --optimal mode will enter my everincoming roster of textual decompressors, speaking of decompression, please consider making a standalone decompressor such as 'PKUnzipJR' - to be easier to compile and benchmark, my wish is to compile it with ICL 64bit. I saw an unofficial binary on encode.ru, quickly tried it with one small textual corpus - the subtitles of all 4 seasons of Lexx superseries:

lexx

AFAIK, the best compression ratio and best decompression speed is held by Hamid's LzTurbo 29, with no entropy coding! On i5-7200u the abridged showdown looks like this:

lexx_subtitles tar

The quick micro-roster:

  624,576 LEXX_subtitles.tar.512M.rz          ! rz_1.01.exe a -d 512M !
  854,296 LEXX_subtitles.tar.zst              ! zstd-v1.3.4-win64.exe --ultra -22 --zstd=wlog=29,clog=30,hlog=30,slog=26 !
1,033,022 LEXX_subtitles.tar.lzt              ! lzturbo 29 !
1,085,000 LEXX_subtitles.tar.optimal_b200.blz ! brieflz_x64.exe --optimal -b200m  !
1,100,568 LEXX_subtitles.tar.9_b200.blz       ! brieflz_x64.exe -9 -b200m !
1,120,775 LEXX_subtitles.tar.optimal.blz
1,135,782 LEXX_subtitles.tar.9.blz
3,535,360 LEXX_subtitles.tar

The TurboBench abridged roster:

Note: For Oodle - 1,085,952 oo2core_6_win64.dll was used.

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>"turbobench_v18.05_-_build_04_may_2018.exe" LEXX_subtitles.tar -elz4,1,16/snappy_c/yappy/zlib,1,5,9/bzip2/oodle,19,49,89,112,114,116,118,129,132,139/lzturbo,19,12,10,29,22,20,39,32,30,49,59,59t2,59t4/brotli,11d29 -I3 -J31 -k1 -B2G
TurboBench:  - Fri Oct 26 23:56:21 2018
      C Size  ratio%     C MB/s     D MB/s   Name            File

      624576               N.A.       N.A.   rz_1.01.exe a -d 512M ! outside TurboBench !

      703293    19.9      12.12      41.99   lzturbo 59                       LEXX_subtitles.tar
      704049    19.9      13.37      59.01   lzturbo 59t2                     LEXX_subtitles.tar
      705289    19.9      13.84      69.61   lzturbo 59t4                     LEXX_subtitles.tar
      727080    20.6       1.23      71.00   lzturbo 49                       LEXX_subtitles.tar
      772881    21.9      13.07      25.73   bzip2                            LEXX_subtitles.tar
      787303    22.3       0.45     308.36   brotli 11d29                     LEXX_subtitles.tar
      803432    22.7       0.45     631.99   oodle 139                        LEXX_subtitles.tar
      816771    23.1       0.68    1062.31   oodle 89                         LEXX_subtitles.tar
      823208    23.3       0.26    1126.63   oodle 129                        LEXX_subtitles.tar
      851330    24.1       1.61    1127.35   lzturbo 39                       LEXX_subtitles.tar

      854296               N.A.       N.A.   zstd-v1.3.4-win64.exe --ultra -22 --zstd=wlog=29,clog=30,hlog=30,slog=26 ! outside TurboBench !

      871033    24.6       0.42     438.68   oodle 19                         LEXX_subtitles.tar
     1008913    28.5      29.17    1138.97   lzturbo 32                       LEXX_subtitles.tar
     1033022    29.2       1.64    1680.30   lzturbo 29                       LEXX_subtitles.tar

     1085000               N.A.       N.A.   brieflz_x64.exe --optimal -b200m ! outside TurboBench !

     1140398    32.3      42.11     588.54   oodle 132                        LEXX_subtitles.tar

     1166389               N.A.    1147.00   Nakamichi 'Ryuugan-ditto-1TB' ! outside TurboBench !

     1186786    33.6       1.83    3003.70   oodle 118                        LEXX_subtitles.tar
     1186786    33.6       1.79    2991.00   oodle 116                        LEXX_subtitles.tar
     1230332    34.8     206.73     863.34   lzturbo 30                       LEXX_subtitles.tar
     1263926    35.8       7.12     230.72   zlib 9                           LEXX_subtitles.tar
     1272951    36.0       4.59    2682.37   lz4 16                           LEXX_subtitles.tar
     1273387    36.0       1.97    3363.81   lzturbo 19                       LEXX_subtitles.tar
     1275499    36.1      23.76     229.02   zlib 5                           LEXX_subtitles.tar
     1306760    37.0      21.84    3039.86   oodle 114                        LEXX_subtitles.tar
     1340179    37.9      30.13    1146.36   lzturbo 22                       LEXX_subtitles.tar
     1391537    39.4      48.35    3383.12   lzturbo 12                       LEXX_subtitles.tar
     1457655    41.2      57.60     235.71   zlib 1                           LEXX_subtitles.tar
     1462916    41.4      76.09    2141.34   oodle 49                         LEXX_subtitles.tar
     1568930    44.4     102.33    3288.71   oodle 112                        LEXX_subtitles.tar
     1764469    49.9     291.96     986.98   lzturbo 20                       LEXX_subtitles.tar
     1866573    52.8      72.12    1371.89   yappy                            LEXX_subtitles.tar
     1896062    53.6     314.34    2828.29   lzturbo 10                       LEXX_subtitles.tar
     1925884    54.5     359.14    2860.32   lz4 1                            LEXX_subtitles.tar
     1937527    54.8     287.94     845.58   snappy_c                         LEXX_subtitles.tar

The lzbench mini-roster:

lzbench 1.7.3 (64-bit Windows)   Assembled by P.Skibinski
Compressor name         Compress. Decompress.  Orig. size  Compr. size  Ratio Filename
memcpy                  14957 MB/s 15683 MB/s     3535360      3535360 100.00 LEXX_subtitles.tar
The results sorted by column number 4:
Compressor name         Compress. Decompress.  Orig. size  Compr. size  Ratio Filename
lzma 16.04 -9            1.71 MB/s    68 MB/s     3535360       761729  21.55 LEXX_subtitles.tar
lzham24 1.0 -4           1.33 MB/s   175 MB/s     3535360       806865  22.82 LEXX_subtitles.tar
tornado 0.6a -16         2.12 MB/s   169 MB/s     3535360       859161  24.30 LEXX_subtitles.tar
xpack 2016-06-02 -9      7.51 MB/s   744 MB/s     3535360      1036273  29.31 LEXX_subtitles.tar
crush 1.0 -2             0.22 MB/s   279 MB/s     3535360      1064067  30.10 LEXX_subtitles.tar

brieflz_x64.exe --optimal -b200m                               1085000  ! outside lzbench !

lzsse2 2016-05-14 -17    1.72 MB/s  3341 MB/s     3535360      1142721  32.32 LEXX_subtitles.tar
lzfse 2017-03-08           48 MB/s   566 MB/s     3535360      1144921  32.38 LEXX_subtitles.tar
lzsse4 2016-05-14 -17    1.91 MB/s  3447 MB/s     3535360      1148438  32.48 LEXX_subtitles.tar
lzsse8 2016-05-14 -17    1.83 MB/s  3169 MB/s     3535360      1170537  33.11 LEXX_subtitles.tar
libdeflate 0.7 -12       5.73 MB/s   561 MB/s     3535360      1186344  33.56 LEXX_subtitles.tar
lzo1b 2.09 -999          8.52 MB/s   516 MB/s     3535360      1273271  36.02 LEXX_subtitles.tar
fastlz 0.1 -2             222 MB/s   371 MB/s     3535360      1794711  50.76 LEXX_subtitles.tar
lzf 3.6 -1                224 MB/s   446 MB/s     3535360      1796011  50.80 LEXX_subtitles.tar
blosclz 2015-11-10 -9     189 MB/s   602 MB/s     3535360      1858900  52.58 LEXX_subtitles.tar
slz_zlib 1.0.0 -3         146 MB/s   193 MB/s     3535360      1894265  53.58 LEXX_subtitles.tar

The Zstd -22..22 modes roster:

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b1e22 --fast -i9 --priority=rt "LEXX_subtitles.tar"
Benchmarking levels from -1 to 22
Note : switching to real-time priority
-1#EXX_subtitles.tar :   3535360 ->   1593113 (2.219), 267.9 MB/s , 806.9 MB/s
 1#EXX_subtitles.tar :   3535360 ->   1259554 (2.807), 245.3 MB/s , 689.9 MB/s
 2#EXX_subtitles.tar :   3535360 ->   1148436 (3.078), 169.5 MB/s , 614.5 MB/s
 3#EXX_subtitles.tar :   3535360 ->   1097874 (3.220), 147.1 MB/s , 606.6 MB/s
 4#EXX_subtitles.tar :   3535360 ->   1085219 (3.258), 136.8 MB/s , 604.7 MB/s
 5#EXX_subtitles.tar :   3535360 ->   1057353 (3.344),  93.2 MB/s , 570.4 MB/s
 6#EXX_subtitles.tar :   3535360 ->   1029604 (3.434),  57.4 MB/s , 606.7 MB/s
 7#EXX_subtitles.tar :   3535360 ->   1014808 (3.484),  38.5 MB/s , 623.9 MB/s
 8#EXX_subtitles.tar :   3535360 ->   1000726 (3.533),  28.7 MB/s , 650.6 MB/s
 9#EXX_subtitles.tar :   3535360 ->    995583 (3.551),  22.6 MB/s , 653.4 MB/s
10#EXX_subtitles.tar :   3535360 ->    985804 (3.586),  14.7 MB/s , 675.0 MB/s
11#EXX_subtitles.tar :   3535360 ->    981015 (3.604),  10.6 MB/s , 672.6 MB/s
12#EXX_subtitles.tar :   3535360 ->    972458 (3.635),  6.48 MB/s , 681.6 MB/s
13#EXX_subtitles.tar :   3535360 ->    963924 (3.668),  6.30 MB/s , 686.5 MB/s
14#EXX_subtitles.tar :   3535360 ->    959726 (3.684),  5.35 MB/s , 695.2 MB/s
15#EXX_subtitles.tar :   3535360 ->    951781 (3.714),  4.10 MB/s , 684.0 MB/s
16#EXX_subtitles.tar :   3535360 ->    871685 (4.056),  3.74 MB/s , 653.8 MB/s
17#EXX_subtitles.tar :   3535360 ->    870377 (4.062),  3.20 MB/s , 644.2 MB/s
18#EXX_subtitles.tar :   3535360 ->    865285 (4.086),  2.77 MB/s , 640.7 MB/s
19#EXX_subtitles.tar :   3535360 ->    862057 (4.101),  2.20 MB/s , 651.7 MB/s
20#EXX_subtitles.tar :   3535360 ->    856131 (4.129),  1.99 MB/s , 632.6 MB/s
21#EXX_subtitles.tar :   3535360 ->    855885 (4.131),  1.99 MB/s , 636.6 MB/s
22#EXX_subtitles.tar :   3535360 ->    854275 (4.138),  1.48 MB/s , 648.3 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=1 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-1#EXX_subtitles.tar :   3535360 ->   1593113 (2.219), 268.0 MB/s , 807.1 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=2 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-2#EXX_subtitles.tar :   3535360 ->   1774908 (1.992), 324.3 MB/s , 850.1 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=3 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-3#EXX_subtitles.tar :   3535360 ->   1886944 (1.874), 349.5 MB/s , 869.6 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=4 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-4#EXX_subtitles.tar :   3535360 ->   1942614 (1.820), 382.8 MB/s , 913.9 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=5 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-5#EXX_subtitles.tar :   3535360 ->   2021020 (1.749), 424.4 MB/s , 963.3 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=6 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-6#EXX_subtitles.tar :   3535360 ->   2140118 (1.652), 450.7 MB/s ,1021.3 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=7 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-7#EXX_subtitles.tar :   3535360 ->   2219871 (1.593), 477.8 MB/s ,1101.5 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=8 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-8#EXX_subtitles.tar :   3535360 ->   2279679 (1.551), 496.7 MB/s ,1111.9 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=9 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-9#EXX_subtitles.tar :   3535360 ->   2360142 (1.498), 517.7 MB/s ,1122.0 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=10 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-10#XX_subtitles.tar :   3535360 ->   2435478 (1.452), 544.4 MB/s ,1168.8 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=11 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-11#XX_subtitles.tar :   3535360 ->   2505871 (1.411), 559.8 MB/s ,1208.8 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=12 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-12#XX_subtitles.tar :   3535360 ->   2623316 (1.348), 602.4 MB/s ,1314.9 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=13 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-13#XX_subtitles.tar :   3535360 ->   2723831 (1.298), 630.6 MB/s ,1475.1 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=14 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-14#XX_subtitles.tar :   3535360 ->   2765710 (1.278), 662.9 MB/s ,1561.5 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=15 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-15#XX_subtitles.tar :   3535360 ->   2821190 (1.253), 694.4 MB/s ,1609.5 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=16 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-16#XX_subtitles.tar :   3535360 ->   2838645 (1.245), 743.5 MB/s ,1744.6 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=17 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-17#XX_subtitles.tar :   3535360 ->   2854756 (1.238), 765.7 MB/s ,1801.4 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=18 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-18#XX_subtitles.tar :   3535360 ->   2874330 (1.230), 786.5 MB/s ,1854.7 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=19 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-19#XX_subtitles.tar :   3535360 ->   2915321 (1.213), 814.1 MB/s ,1938.2 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=20 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-20#XX_subtitles.tar :   3535360 ->   2939904 (1.203), 847.2 MB/s ,2027.4 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=21 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-21#XX_subtitles.tar :   3535360 ->   2916898 (1.212), 853.8 MB/s ,2072.7 MB/s

E:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=22 -i9 --priority=rt "LEXX_subtitles.tar"
Note : switching to real-time priority
-22#XX_subtitles.tar :   3535360 ->   2917589 (1.212), 853.5 MB/s ,2082.6 MB/s

If you are interested, I have one superb corpus - a definitive one for Japanese language - revealing how strong one parser is in ~190MB depths, could post how BriefLZ performs...

kenk22222222motoiglo-img1200x900-1507974451v5ppgf10451

jibsen commented 6 years ago

Thanks for the interesting results, I am happy to see BriefLZ is doing okay.

Regarding ICL, if it is compatible with the command line options of MSVC cl, I imagine you could do

set CC=icl
nmake -f Makefile.vc

in the example folder to build blzpack.

You could also simply list all the source files -- something like:

icl /O2 /I..\include blzpack.c parg.c ..\src\brieflz.c ..\src\depack.c ..\src\depacks.c

Replace /O2 with whatever optimization options ICL provides.

Sanmayce commented 6 years ago

Thank you, glad that I avoided automated 'make' and such, I totally rely on manual (command line) compilation.

The x86 and x64 compiles are attached: BriefLZ_120_Intel_v15_32bit.exe.zip BriefLZ_120_Intel_v15_64bit.exe.zip

Intel(R) Parallel Studio XE 2015
Copyright (C) 1985-2014 Intel Corporation. All rights reserved.
Intel(R) Parallel Studio XE 2015 Composer Edition (package 108)
Setting environment for using Microsoft Visual Studio 2010 x64 cross tools.

C:\Program Files\Intel\Composer XE 2015>d:

D:\>cd brieflz-1.2.0

D:\brieflz-1.2.0>dir

10/27/2018  09:10 PM    <DIR>          .
10/27/2018  09:10 PM    <DIR>          ..
10/25/2018  08:24 PM                62 .gitignore
10/25/2018  08:24 PM             1,552 .travis.yml
10/25/2018  08:24 PM               868 appveyor.yml
10/25/2018  08:24 PM             6,004 CMakeLists.txt
10/25/2018  08:24 PM               354 Doxyfile
10/27/2018  09:01 PM    <DIR>          example
10/25/2018  08:24 PM    <DIR>          include
10/25/2018  08:24 PM               903 LICENSE
10/27/2018  09:00 PM               165 makeEXE.bat
10/25/2018  08:24 PM             1,046 NEWS
10/25/2018  08:24 PM             4,530 README.md
10/25/2018  08:24 PM    <DIR>          src
10/25/2018  08:24 PM    <DIR>          test

D:\brieflz-1.2.0>makeEXE.bat

D:\brieflz-1.2.0>cd example

D:\brieflz-1.2.0\example>icl /Qstd=c99 /Ox /I..\include blzpack.c parg.c ..\src\brieflz.c ..\src\depack.c ..\src\depacks.c
Intel(R) C++ Compiler XE for applications running on Intel(R) 64, Version 15.0.0.108 Build 20140726
Copyright (C) 1985-2014 Intel Corporation.  All rights reserved.

blzpack.c
parg.c
brieflz.c
depack.c
depacks.c
Microsoft (R) Incremental Linker Version 10.00.40219.01
Copyright (C) Microsoft Corporation.  All rights reserved.

-out:blzpack.exe
blzpack.obj
parg.obj
brieflz.obj
depack.obj
depacks.obj

D:\brieflz-1.2.0\example>copy blzpack.exe ..\BriefLZ_120_Intel_v15_64bit.exe /y
        1 file(s) copied.

D:\brieflz-1.2.0\example>cd..

D:\brieflz-1.2.0>dir bri*.exe

10/27/2018  09:01 PM           136,192 BriefLZ_120_Intel_v15_32bit.exe
10/27/2018  09:10 PM           144,384 BriefLZ_120_Intel_v15_64bit.exe

D:\brieflz-1.2.0>

The results for the two Kenkyusha are under way...

Sanmayce commented 6 years ago

Having looked into your code, very clean-n-simple, my appreciation for BriefLZ reaches new heights, thanks for your work, no matter how much time the compression takes I am going... By the way, one suggestion from me, consider dedicating an old laptop with working battery and just throw enwik9 at your supertoy, leave it ... aloof, hee-hee for a year if you must and shock the [de]compression community :P

For 3 days my i5-7200u has been crunching the 190MB testset, progress indicator is needed, so I added one, printing the 'cur' from brieflz_ssparse.h does the feedback.:

    // Phase 2: Find lowest cost path from each position to end
    for (unsigned long cur = last_match_pos; cur > 0; --cur) {

        printf("%lu             \r", cur); //Kaze

        // Since we updated prev to the end in the first phase, we
        // do not need to hash, but can simply look up the previous
        // position directly.

Hope you don't mind my new compile allowing benchmarking decompression rates, all lines which I added end in //Kaze, the fragment in blzpack.c in decompress_file function:

for (ik = 1; ik <= 16; ik++) { //Kaze
fseek(packedfile, 0, SEEK_SET); //Kaze

    clocks = clock();
    clocksWITHOUTfreadTotal = 0; //Kaze
    insize = 0; outsize =0; //Kaze

    /* While we are able to read a header from input file .. */
    while (fread(header, 1, sizeof(header), packedfile) == sizeof(header)) {
        size_t hdr_packedsize, hdr_depackedsize, depackedsize;
        unsigned long crc;

        /* Show a little progress indicator */
        if (be_verbose) {
            fprintf(stderr, "%c\r", rotator[counter]);
            counter = (counter + 1) & 0x03;
        }

        /* Get compressed and original size from header */
        hdr_packedsize = (size_t) read_be32(header + 2 * 4);
        hdr_depackedsize = (size_t) read_be32(header + 4 * 4);

        /* Verify values in header */
        if (read_be32(header + 0 * 4) != 0x626C7A1AUL /* "blz\x1A" */
         || read_be32(header + 1 * 4) != 1) {
            printf_error("invalid header in compressed file");
            goto out;
        }

        /* Check blocksize is sufficient */
        if (hdr_packedsize > max_packed_size
         || hdr_depackedsize > blocksize) {
            printf_usage("compressed file requires block size"
                     " >= %lu bytes", hdr_depackedsize);
            goto out;
        }

        /* Read compressed data */
        if (fread(packed, 1, hdr_packedsize, packedfile) != hdr_packedsize) {
            printf_error("error reading block from compressed file");
            goto out;
        }

        /* Check CRC32 of compressed data */
        crc = read_be32(header + 3 * 4);
        if (use_checksum
         && crc != 0
         && crc != blz_crc32(packed, hdr_packedsize, 0)) {
            printf_error("compressed data crc error");
            goto out;
        }

    clocksWITHOUTfread = clock(); //Kaze

        /* Decompress data */
        if (use_safe) {
            depackedsize = blz_depack_safe(
                    packed, (unsigned long) hdr_packedsize,
                    data, (unsigned long) hdr_depackedsize);
        } else {
            depackedsize = blz_depack(packed, data,
                    (unsigned long) hdr_depackedsize);
        }

        /* Check for decompression error */
        if (depackedsize != hdr_depackedsize) {
            printf_error("an error occured while decompressing");
            goto out;
        }

        /* Check CRC32 of decompressed data */
        crc = read_be32(header + 5 * 4);
        if (use_checksum
         && crc != 0
         && crc != blz_crc32(data, depackedsize, 0)) {
            printf_error("decompressed data crc error");
            goto out;
        }

        /* Write decompressed data */
//      fwrite(data, 1, depackedsize, newfile); //Kaze

        /* sum input and output size */
        insize += (unsigned long) (hdr_packedsize + sizeof(header));
        outsize += (unsigned long) depackedsize;

    clocksWITHOUTfreadTotal += clock() - clocksWITHOUTfread; //Kaze

    }

    k = (((double)CLOCKS_PER_SEC*outsize/(double)clocksWITHOUTfreadTotal)); k=k>>20;  //Kaze
    if (Kmax<k) Kmax=k; //Kaze
    printf("RAM-to-RAM performance: %d MB/s.\n", k); //Kaze

    clocks = clock() - clocks;

} //Kaze

    printf("RAM-to-RAM (PEAK) performance: %d MB/s.\n", Kmax); //Kaze

brieflz-1.2.0_Intel_Bench.zip

Oh, and here comes an excerpt of the two concatenated Kenkyusha, hate when the reader is not allowed to feel the testdatafile:

    "charCode": "jisx0208",
    "discCode": "epwing",
    "subbooks": [
        {
            "title": "研究社　新英和大辞典　第６版",
            "copyright": "研究社　新英和大辞典 第６版 (EPWING CD-ROM 版)\n(C)2006　株式会社研究社\n\n本ディスクに収録されたデータは著作権法によって保護されており,個人として使用する場合のほかは著作権者に無断で転載・複製することはできません.\n\n",
            "entries": [
...
                {
                    "heading": "spill light",
                    "text": "sp{{n_41307}}ll l{{n_41334}}ght\n{{w_45349}}n. 【劇場】 ＝spill１ 3 a.\n"
                },
                {
                    "heading": "spill one's guts",
                    "text": "sp{{n_41307}}ll one's g{{n_41309}}ts 《米口語》 すべてを打ち明ける.\n"
                },
                {
                    "heading": "spill one's guts",
                    "text": "sp{{n_41307}}ll one's g{{n_41309}}ts 《俗》 腹のうちを吐き出す, 何もかもぶちまける.\n"
                },
                {
                    "heading": "spill over",
                    "text": "sp{{n_41307}}ll {{n_41308}}ver あふれ出る, あふれて流れる.\n"
                },
                {
                    "heading": "spillover",
                    "text": "sp{{n_41307}}ll・{{n_41335}}ver\n{{w_45349}}n.\n1 こぼすこと.\n2 《米》 こぼれた[あふれ出た]もの; 過剰.\n3 【経済】 溢出(いっしゅつ)効果《公共支出による間接的影響》.\n{{n_42344}}1920{{n_42345}}\n"
                }
            ]
        }
    ]
}

    "charCode": "jisx0208",
    "discCode": "epwing",
    "subbooks": [
        {
            "title": "研究社　新和英大辞典　第５版",
            "copyright": "研究社　新和英大辞典 第５版 (EPWING CD-ROM 版)\n(C)2004　株式会社研究社\n\n本ディスクに収録されたデータは著作権法によって保護されており,個人として使用する場合のほかは著作権者に無断で転載・複製することはできません.\n\n",
            "entries": [
                {
                    "heading": "外字一覧",
                    "text": "外字一覧\n半角\n全角\n　\n　\n"
                },
                {
                    "heading": "〆鯖",
                    "text": "しめさば【締め鯖・〆鯖】 {{w_46695}}(shimesaba)\n【料理】 (pieces of) salted and vinegared mackerel.\n"
                },
                {
                    "heading": "090金融",
                    "text": "ゼロきゅうゼロきんゆう【090金融】 {{w_46695}}(zeroky{{n_41529}}zerokin'y{{n_41529}})\n〔携帯電話を使う無店舗貸金業〕 \"090\" [cellphone] financing.\n"
                },
                {
                    "heading": "¶0 [ゼロ]発信 ＜はっしん１【発信】＞",
                    "text": "{{w_46700}}0 [ゼロ]発信\n"
                },
                {
                    "heading": "¶1,000円紙幣 ＜しへい２【紙幣】＞",
                    "text": "{{w_46686}}1,000 円紙幣　a thousand-yen {{w_45629}}{{n_41269}}bill [{{n_41270}}note]\n"
                },
...
                {
                    "heading": "¶ジグザグ・ミシン ＜ジグザグ＞",
                    "text": "ジグザグ・ミシン　a zigzag sewing machine.\nじくじ【忸怩】 {{w_46695}}(jikuji)\n"
                },
                {
                    "heading": "¶ジグザグ・デモ ＜ジグザグ＞",
                    "text": "ジグザグ・デモ　a snakedance (demonstration).\n"
                },
                {
                    "heading": "¶ジグザグ・コース ＜ジグザグ＞",
                    "text": "ジグザグ・コース\n"
                }
            ]
        }
    ]
}

As for ongoing 190MB the current results are:

fv

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>lzbench173 -c4 -i1,15 -o3 -etornado,16/blosclz,9/brieflz/crush,2/csc,5/density,3/fastlz,2/gipfeli/zstd24,1,12,22/lzo1b,999/lzham,4/lzham24,4/libdeflate,1,12/lz4fast,1,99/lz4hc,1,10,12/lizard,19,29,39,49/lzf,1/lzfse/lzg,9/lzham,1/lzjb/lzlib,9/lzma,9/lzrw,5/lzsse2,17/lzsse4,17/lzsse8,17/lzvn/pithy,9/quicklz,3/snappy/slz_zlib,3/ucl_nrv2b,9/ucl_nrv2d,9/ucl_nrv2e,9/xpack,1,9/xz,9/yalz77,12/yappy,99/zlib,1,5,9/zling,4/shrinker/wflz/lzmat "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar"
lzbench 1.7.3 (64-bit Windows)   Assembled by P.Skibinski
Compressor name         Compress. Decompress.  Orig. size  Compr. size  Ratio Filename
memcpy                  11274 MB/s 11166 MB/s   192712192    192712192 100.00 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
...
The results sorted by column number 4:
Compressor name         Compress. Decompress.  Orig. size  Compr. size  Ratio Filename
lzma 16.04 -9            1.58 MB/s   103 MB/s   192712192     31946702  16.58 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
xz 5.2.3 -9              1.71 MB/s   101 MB/s   192712192     31946936  16.58 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
tornado 0.6a -16         1.83 MB/s   235 MB/s   192712192     32206812  16.71 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzham 1.0 -d26 -4        1.06 MB/s   253 MB/s   192712192     32317346  16.77 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzlib 1.8 -9             1.30 MB/s    71 MB/s   192712192     32384613  16.80 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
csc 2016-10-13 -5        2.01 MB/s    92 MB/s   192712192     32999985  17.12 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
zstd24 1.3.3 -22         1.89 MB/s   636 MB/s   192712192     34560949  17.93 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzham24 1.0 -4           1.22 MB/s   245 MB/s   192712192     34793710  18.05 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
zling 2016-01-10 -4        28 MB/s   156 MB/s   192712192     37539507  19.48 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzham 1.0 -d26 -1        2.24 MB/s   237 MB/s   192712192     38781141  20.12 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

BriefLZ 1.2.0 -9 -b200              N.A. MB/s                 39275702        ! Outside lzbench !

zstd24 1.3.3 -12           13 MB/s   708 MB/s   192712192     42129821  21.86 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
crush 1.0 -2             0.34 MB/s   330 MB/s   192712192     42804848  22.21 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

Nakamichi 'Ryuugan-ditto-1TB'       1038 MB/s                 43436425        ! Outside lzbench !

xpack 2016-06-02 -9        14 MB/s   914 MB/s   192712192     47148645  24.47 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
libdeflate 0.7 -12       4.25 MB/s   699 MB/s   192712192     49121792  25.49 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lizard 1.0 -49           1.60 MB/s  1251 MB/s   192712192     49333394  25.60 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lizard 1.0 -29           1.68 MB/s  1428 MB/s   192712192     50185897  26.04 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzfse 2017-03-08           53 MB/s   724 MB/s   192712192     50528082  26.22 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
zlib 1.2.11 -9             19 MB/s   296 MB/s   192712192     51106894  26.52 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
ucl_nrv2e 1.03 -9        1.75 MB/s   298 MB/s   192712192     51333113  26.64 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
ucl_nrv2d 1.03 -9        1.67 MB/s   298 MB/s   192712192     51725356  26.84 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
zlib 1.2.11 -5             40 MB/s   289 MB/s   192712192     52408484  27.20 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
ucl_nrv2b 1.03 -9        1.87 MB/s   290 MB/s   192712192     52696273  27.34 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

LZSSE2 17 AVX2 Intel                3436 MB/s                 53939869        ! Outside lzbench !

lzsse2 2016-05-14 -17    5.52 MB/s  3319 MB/s   192712192     53939869  27.99 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzsse4 2016-05-14 -17    5.97 MB/s  3558 MB/s   192712192     54910344  28.49 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzsse8 2016-05-14 -17    5.76 MB/s  3470 MB/s   192712192     55238484  28.66 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
xpack 2016-06-02 -1       138 MB/s   763 MB/s   192712192     55818895  28.96 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
libdeflate 0.7 -1         147 MB/s   698 MB/s   192712192     56297784  29.21 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzmat 1.01                 38 MB/s   295 MB/s   192712192     56988613  29.57 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lz4hc 1.8.0 -12          6.46 MB/s  2420 MB/s   192712192     57326349  29.75 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lizard 1.0 -19           4.23 MB/s  2804 MB/s   192712192     57394968  29.78 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzo1b 2.09 -999            14 MB/s   550 MB/s   192712192     57751742  29.97 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lz4hc 1.8.0 -10            24 MB/s  2407 MB/s   192712192     57829440  30.01 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lizard 1.0 -39           4.10 MB/s  2651 MB/s   192712192     58035876  30.12 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzg 1.0.8 -9             0.76 MB/s   517 MB/s   192712192     58945777  30.59 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
brieflz 1.1.0             105 MB/s   158 MB/s   192712192     59644390  30.95 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
zlib 1.2.11 -1             83 MB/s   277 MB/s   192712192     60050933  31.16 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
zstd24 1.3.3 -1           287 MB/s   766 MB/s   192712192     60306373  31.29 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
quicklz 1.5.0 -3           45 MB/s   704 MB/s   192712192     60684066  31.49 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
yalz77 2015-09-19 -12      22 MB/s   313 MB/s   192712192     60801317  31.55 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lz4hc 1.8.0 -1             98 MB/s  2304 MB/s   192712192     61109730  31.71 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzvn 2017-03-08            46 MB/s   836 MB/s   192712192     61122683  31.72 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
gipfeli 2016-07-13        248 MB/s   359 MB/s   192712192     66237309  34.37 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
pithy 2011-12-24 -9       255 MB/s  1380 MB/s   192712192     68003260  35.29 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
density 0.12.5 beta -3    257 MB/s   267 MB/s   192712192     69457934  36.04 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzrw 15-Jul-1991 -5       114 MB/s   419 MB/s   192712192     70484288  36.57 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
yappy 2014-03-22 -99       68 MB/s  1973 MB/s   192712192     73104262  37.93 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
slz_zlib 1.0.0 -3         198 MB/s   250 MB/s   192712192     74118745  38.46 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzf 3.6 -1                269 MB/s   547 MB/s   192712192     75365164  39.11 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
fastlz 0.1 -2             242 MB/s   438 MB/s   192712192     75979538  39.43 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lz4fast 1.8.0 -1          479 MB/s  2376 MB/s   192712192     76930096  39.92 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
blosclz 2015-11-10 -9     231 MB/s   792 MB/s   192712192     77134819  40.03 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
snappy 1.1.4              323 MB/s  1097 MB/s   192712192     78886328  40.93 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
wflz 2015-09-16           208 MB/s   685 MB/s   192712192     85850692  44.55 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lzjb 2010                 254 MB/s   434 MB/s   192712192     94276864  48.92 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
shrinker 0.1              972 MB/s  2466 MB/s   192712192    153287483  79.54 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
lz4fast 1.8.0 -99        1868 MB/s  4634 MB/s   192712192    162174357  84.15 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>"turbobench_v18.05_-_build_04_May_2018.exe" "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar" -elz4,1,16/snappy_c/yappy/zlib,1,5,9/bzip2/lzlib,9d29fb273/lzham,4fb258:x4:d29/lzma,9d29:fb273:mf=bt4/oodle,19,49,89,112,114,116,118,129,132,139/lzturbo,19,12,10,29,22,20,39,32,30,49,59,59t2,59t4/zstd,1,5,12,22,22d29/lizard,11,19,21,29,31,39,41,49/brotli,1,5,11/brotli,11d29/lzma,9/chameleon,2/density,3/lzham,4/trle/bsc,3,6/zpaq,2,5 -I3 -J31 -k1 -B2G
TurboBench:  - Sat Oct 27 09:35:36 2018

      C Size  ratio%     C MB/s     D MB/s   Name            File
    24583848    12.8       0.39       0.39   zpaq 5                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    25678685    13.3      10.28      41.03   lzturbo 59                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    25680857    13.3      10.85      55.92   lzturbo 59t2                     Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    25683861    13.3      10.99      60.89   lzturbo 59t4                     Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    27222502    14.1      24.19      10.07   bsc 6                            Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    30040086    15.6       0.82     107.67   lzturbo 49                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    30619955    15.9       1.17      76.78   lzlib 9d29fb273                  Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    30688469    15.9       1.27     108.58   lzma 9d29:fb273:mf=bt4           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    30750929    16.0       0.42     366.75   brotli 11d29                     Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    30994988    16.1       0.29     262.69   lzham 4fb258:x4:d29              Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31215811    16.2       1.00     864.77   lzturbo 39                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31613856    16.4       1.57     720.23   zstd 22d29                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31700709    16.4       1.60     720.34   zstd 22                          Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31736080    16.5       0.27     638.93   oodle 139                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31828542    16.5       1.13     255.50   lzham 4                          Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31856985    16.5       0.19     743.89   oodle 129                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31857002    16.5       0.33     743.94   oodle 89                         Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31946706    16.6       1.60     104.95   lzma 9                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    31998098    16.6      29.28      16.18   bsc 3                            Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    33518833    17.4       0.28     417.00   oodle 19                         Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    35191618    18.3      13.00      35.52   bzip2                            Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    35731577    18.5       0.46     417.58   brotli 11                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    36266918    18.8       1.01    1222.23   lzturbo 29                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    38296917    19.9      43.83     958.75   lzturbo 32                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

    39275702                          N.A.   BriefLZ 1.2.0 -9 -b200           ! Outside TurboBench !

    42129825    21.9      11.96     944.08   zstd 12                          Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    42283586    21.9       5.02      84.39   zpaq 2                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    42368723    22.0      24.77     415.24   brotli 5                         Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

    43436425                       1038.00   Nakamichi 'Ryuugan-ditto-1TB'    ! Outside TurboBench !

    46192148    24.0       1.64    1050.32   lizard 49                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    48078069    24.9      91.19     944.80   zstd 5                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    49818924    25.9      41.93    1146.22   lzturbo 22                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    50185901    26.0       1.73    1527.86   lizard 29                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    50353157    26.1      42.37     642.64   oodle 132                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    51106898    26.5      19.06     287.97   zlib 9                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    51500191    26.7       0.56    2292.04   oodle 118                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    51500191    26.7       0.56    2262.97   oodle 116                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    52408488    27.2      37.09     281.22   zlib 5                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    52906608    27.5       4.36    1918.20   lizard 39                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    53861681    27.9     217.52    1096.24   lzturbo 30                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

    53939869                       3436.00   LZSSE2 17 AVX2 Intel             ! Outside TurboBench !

    56652076    29.4     165.57     354.09   brotli 1                         Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    57326353    29.7       5.72    2825.36   lz4 16                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    57350396    29.8       1.21    3267.53   lzturbo 19                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    57394972    29.8       4.69    2686.26   lizard 19                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    60050937    31.2      70.03     269.59   zlib 1                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    60306377    31.3     296.39     913.30   zstd 1                           Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    60874329    31.6     148.21    1036.16   lizard 41                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    60943720    31.6      31.25    3273.19   oodle 114                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    61263659    31.8      51.95    3253.41   lzturbo 12                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    63102351    32.7      97.01    2103.43   oodle 49                         Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    66434556    34.5     197.91    1887.61   lizard 31                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    69003330    35.8      85.52    2843.96   oodle 112                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    69203282    35.9     183.62    1925.54   lizard 21                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    69457210    36.0     258.40     285.57   density 3                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    70671326    36.7     246.50    2675.18   lizard 11                        Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    72257510    37.5     396.23    1346.03   lzturbo 20                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    74310545    38.6      85.29    2070.66   yappy                            Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    76266546    39.6     444.50    3031.21   lzturbo 10                       Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    76930100    39.9     470.87    2887.16   lz4 1                            Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
    78877366    40.9     377.70    1098.90   snappy_c                         Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
   115124486    59.7    1605.61    2235.10   chameleon 2                      Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
   162256560    84.2     221.78    2066.49   trle                             Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

Will add the decompression speeds after finishing the -x -b200 ...

10/27/2018  04:00 AM       192,712,192 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar
07/15/2018  05:09 PM        53,939,869 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.L17.LZSSE2
10/27/2018  11:23 AM        48,534,916 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.9.blz
10/27/2018  02:14 PM        47,880,557 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.optimal.blz
07/15/2018  05:07 PM        43,436,425 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.Nakamichi
10/27/2018  12:15 PM        39,275,702 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.9_b200.blz
10/27/2018  02:14 PM                 0 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.optimal_b200.blz

The actual invocations:

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 brieflz_x64.exe -9 "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar" "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar".9.blz

Kernel  Time =     0.046 =    0%
User    Time =    35.000 =   99%
Process Time =    35.046 =   99%    Virtual  Memory =     15 MB
Global  Time =    35.207 =  100%    Physical Memory =     16 MB

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 brieflz_x64.exe -9 -b200m "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar" "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar".9_b200.blz

Kernel  Time =     1.359 =    0%
User    Time =  3106.640 =   99%
Process Time =  3108.000 =   99%    Virtual  Memory =   2831 MB
Global  Time =  3108.866 =  100%    Physical Memory =   2445 MB

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 brieflz_x64.exe --optimal "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar" "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar".optimal.blz

Kernel  Time =     0.078 =    0%
User    Time =  7146.156 =   99%
Process Time =  7146.234 =   99%    Virtual  Memory =     15 MB
Global  Time =  7147.836 =  100%    Physical Memory =     16 MB

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 brieflz_x64.exe --optimal -b200m "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar" "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar".optimal_b200.blz
...
jibsen commented 6 years ago

Thank you again for all the details.

One thing I noticed in your blzpack modification is that there are two freads, one to read the block header, and one to read the compressed data. You put the line setting clocksWITHOUTfread between these two. If you want to get the time of decompression only you will have to move it down after the second fread.

Sanmayce commented 6 years ago

> If you want to get the time of decompression only you will have to move it down after the second fread.

Ugh, dummy me, a stupid overlook, now moved right above the actual decompression, edited the previous post and reuploaded the .zip.

Sanmayce commented 6 years ago

After nearly 10 days of crunching, had to terminate the --optimal -b200m, mostly because of the lack of progress indicator - this is unacceptable, so I have to rerun it with my Oct 30 compile.

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 brieflz_x64.exe --optimal -b200m "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar" "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar".optimal_b200.blz

Exit code: -1073741510

Kernel  Time =     2.656 =    0%
User    Time =977418.125 =   99%
Process Time =977420.781 =   99%    Virtual  Memory =   2831 MB
Global  Time =977614.483 =  100%    Physical Memory =   1602 MB
^CTerminate batch job (Y/N)? y

As for the decompression rate:

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>BriefLZ_120_Intel_v15_64bit.exe -v -d -b200m "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.9_b200.blz" k2
Current priority class is REALTIME_PRIORITY_CLASS.
RAM-to-RAM performance: 250 MB/s.
RAM-to-RAM performance: 273 MB/s.
RAM-to-RAM performance: 273 MB/s.
RAM-to-RAM performance: 273 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM (PEAK) performance: 280 MB/s.
in 39275702 out 192712192 ratio 20% time 0.67

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>BriefLZ_120_Intel_v15_64bit.exe -v -d -b200m "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.9_b200.blz" k2
Current priority class is REALTIME_PRIORITY_CLASS.
RAM-to-RAM performance: 261 MB/s.
RAM-to-RAM performance: 273 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 280 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM performance: 279 MB/s.
RAM-to-RAM (PEAK) performance: 280 MB/s.
in 39275702 out 192712192 ratio 20% time 0.67

And for a good measure, Razor and Zstd are added:

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 zstd-v1.3.4-win64.exe --ultra -22 --zstd=wlog=29,clog=30,hlog=30,slog=26 "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar"
Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar : 16.40%   (192712192 => 31614404 bytes, Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.zst)

Kernel  Time =     3.765 =    2%
User    Time =   144.109 =   96%
Process Time =   147.875 =   99%    Virtual  Memory =  18479 MB
Global  Time =   149.101 =  100%    Physical Memory =   7314 MB

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>timer64 rz_1.01.exe a -d 512M "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar".512M.rz "Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar"

 *** RAZOR Archiver 1.01 (2017-09-14) - DEMO/TEST version ***
 *** (c) Christian Martelock (christian.martelock@web.de) ***

 Scanning c:\_textual_madness_bare-minimum_2018-oct-26\kenkyusha_(english-japanese)_(japanese-english)_dictionaries_epwing-to-json.tar
 Found 0 dirs, 1 files, 192712192 bytes.

 Creating archive Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.512M.rz
 Window : 188196K (1024M..256G)
 Header : 102
 Size   : 25067019

 Archive ok. Added 0 dirs, 1 files, 192712192 bytes.
 CPU time = 536.578s / wall time = 429.721s

Kernel  Time =     1.812 =    0%
User    Time =   536.578 =  124%
Process Time =   538.390 =  125%    Virtual  Memory =   2320 MB
Global  Time =   430.217 =  100%    Physical Memory =   2048 MB

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b1e22 --fast -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Benchmarking levels from -1 to 22
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-1#PWING-to-JSON.tar : 192712192 ->  69137271 (2.787), 337.5 MB/s ,1086.5 MB/s
 1#PWING-to-JSON.tar : 192712192 ->  60306373 (3.196), 304.5 MB/s , 825.3 MB/s
 2#PWING-to-JSON.tar : 192712192 ->  54528894 (3.534), 209.9 MB/s , 788.5 MB/s
 3#PWING-to-JSON.tar : 192712192 ->  50499202 (3.816), 174.9 MB/s , 775.9 MB/s
 4#PWING-to-JSON.tar : 192712192 ->  49723919 (3.876), 154.4 MB/s , 767.4 MB/s
 5#PWING-to-JSON.tar : 192712192 ->  48078065 (4.008), 105.2 MB/s , 748.2 MB/s
 6#PWING-to-JSON.tar : 192712192 ->  45548783 (4.231),  68.9 MB/s , 763.7 MB/s
 7#PWING-to-JSON.tar : 192712192 ->  44703350 (4.311),  53.3 MB/s , 780.6 MB/s
 8#PWING-to-JSON.tar : 192712192 ->  44205825 (4.359),  39.4 MB/s , 803.8 MB/s
 9#PWING-to-JSON.tar : 192712192 ->  43799584 (4.400),  32.7 MB/s , 808.4 MB/s
10#PWING-to-JSON.tar : 192712192 ->  43427498 (4.438),  24.8 MB/s , 816.8 MB/s
11#PWING-to-JSON.tar : 192712192 ->  42389667 (4.546),  18.3 MB/s , 752.3 MB/s
12#PWING-to-JSON.tar : 192712192 ->  42129821 (4.574),  12.9 MB/s , 760.7 MB/s
13#PWING-to-JSON.tar : 192712192 ->  41455964 (4.649),  10.1 MB/s , 760.1 MB/s
14#PWING-to-JSON.tar : 192712192 ->  41159037 (4.682),  8.58 MB/s , 765.8 MB/s
15#PWING-to-JSON.tar : 192712192 ->  40305269 (4.781),  6.31 MB/s , 720.2 MB/s
16#PWING-to-JSON.tar : 192712192 ->  39663967 (4.859),  5.15 MB/s , 771.0 MB/s
17#PWING-to-JSON.tar : 192712192 ->  38788384 (4.968),  4.08 MB/s , 720.7 MB/s
18#PWING-to-JSON.tar : 192712192 ->  38135499 (5.053),  3.59 MB/s , 714.5 MB/s
19#PWING-to-JSON.tar : 192712192 ->  36701360 (5.251),  2.89 MB/s , 671.9 MB/s
20#PWING-to-JSON.tar : 192712192 ->  33862705 (5.691),  1.97 MB/s , 641.9 MB/s
21#PWING-to-JSON.tar : 192712192 ->  32459477 (5.937),  1.79 MB/s , 654.2 MB/s
22#PWING-to-JSON.tar : 192712192 ->  31700705 (6.079),  1.63 MB/s , 660.2 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=1 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-1#PWING-to-JSON.tar : 192712192 ->  69137271 (2.787), 335.1 MB/s ,1079.2 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=2 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-2#PWING-to-JSON.tar : 192712192 ->  77484995 (2.487), 373.8 MB/s ,1130.3 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=3 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-3#PWING-to-JSON.tar : 192712192 ->  80258175 (2.401), 399.9 MB/s ,1074.9 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=4 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-4#PWING-to-JSON.tar : 192712192 ->  84855053 (2.271), 408.1 MB/s ,1145.8 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=5 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-5#PWING-to-JSON.tar : 192712192 ->  87649791 (2.199), 378.6 MB/s ,1085.3 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=6 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-6#PWING-to-JSON.tar : 192712192 ->  91026879 (2.117), 452.7 MB/s ,1150.1 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=7 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-7#PWING-to-JSON.tar : 192712192 ->  93330210 (2.065), 387.5 MB/s ,1201.2 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=8 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-8#PWING-to-JSON.tar : 192712192 ->  95850574 (2.011), 412.8 MB/s ,1228.8 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=9 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-9#PWING-to-JSON.tar : 192712192 ->  97877922 (1.969), 457.9 MB/s ,1257.4 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=10 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-10#WING-to-JSON.tar : 192712192 -> 100134568 (1.925), 515.0 MB/s ,1235.5 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=11 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-11#WING-to-JSON.tar : 192712192 -> 102326958 (1.883), 531.1 MB/s ,1346.2 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=12 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-12#WING-to-JSON.tar : 192712192 -> 104549155 (1.843), 546.4 MB/s ,1351.7 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=13 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-13#WING-to-JSON.tar : 192712192 -> 106504920 (1.809), 585.2 MB/s ,1414.8 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=14 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-14#WING-to-JSON.tar : 192712192 -> 108387393 (1.778), 590.4 MB/s ,1451.5 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=15 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-15#WING-to-JSON.tar : 192712192 -> 110039936 (1.751), 617.3 MB/s ,1478.8 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=16 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-16#WING-to-JSON.tar : 192712192 -> 111700796 (1.725), 617.3 MB/s ,1494.7 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=17 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-17#WING-to-JSON.tar : 192712192 -> 113287429 (1.701), 634.2 MB/s ,1502.1 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=18 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-18#WING-to-JSON.tar : 192712192 -> 114779714 (1.679), 627.6 MB/s ,1522.8 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=19 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-19#WING-to-JSON.tar : 192712192 -> 116112784 (1.660), 659.6 MB/s ,1553.6 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=20 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-20#WING-to-JSON.tar : 192712192 -> 117386823 (1.642), 663.1 MB/s ,1563.5 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=21 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-21#WING-to-JSON.tar : 192712192 -> 118456967 (1.627), 694.3 MB/s ,1602.6 MB/s

C:\_TEXTUAL_MADNESS_bare-minimum_2018-Oct-26>zstd-v1.3.4-win64.exe -b --fast=22 -i9 --priority=rt ""Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar""
Note : switching to real-time priority apanese-English)_Dictionaries_EPWING-to-JSON.tar...
-22#WING-to-JSON.tar : 192712192 -> 119229712 (1.616), 705.1 MB/s ,1632.5 MB/s

Razor proves its worth over and over again:

11/07/2018  09:05 PM        25,067,019 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.512M.rz
11/07/2018  08:53 PM        31,614,404 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.512M.zst
10/27/2018  11:15 AM        39,275,702 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.9_b200.blz
07/15/2018  04:07 PM        43,436,425 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.Nakamichi
07/15/2018  04:09 PM        53,939,869 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar.L17.LZSSE2
11/07/2018  08:53 PM       192,712,192 Kenkyusha_(English-Japanese)_(Japanese-English)_Dictionaries_EPWING-to-JSON.tar

Still wanna make a comprehensive i.e unabridged roster with many options and decompressors, don't know when. What could be the first testdatafile... do you have some in mind... my first choice would be The Complete Dostoyevskiy ~40MB, the filesize should be >> L3.