sergey-dryabzhinsky / python-zstd

Simple python bindings to Yann Collet ZSTD compression library
BSD 2-Clause "Simplified" License
165 stars 27 forks source link

Allow use extra-fast levels #56

Closed sergey-dryabzhinsky closed 3 years ago

sergey-dryabzhinsky commented 3 years ago

As on Zstandard page in comparison table - it may be 7 or probably even 100. Research and Discussion needed.

sergey-dryabzhinsky commented 3 years ago

It's safe to use -100 level, at least since 1.4.5. Tested on 12Mb SQL-dump. Speed is greate but ratio is suck.

Compression

Table of ratio in %:
level       zstd        zlib     
    -100       63.74            
     -50       52.61            
     -20       35.06            
     -10       28.78            
      -5       25.64            
      -4       25.10            
      -3       24.73            
      -2       24.16            
      -1       23.78            
       1       17.68       21.80
       2       17.88       21.15
       3       17.67       20.54
       4       18.40       19.19
       5       17.73       18.27
       6       16.99       17.92
       7       16.68       17.76
       8       16.54       17.58
       9       16.46       17.58
      10       16.41            
      11       16.32            
      12       16.30            
      13       15.87            
      14       14.79            
      15       14.33            
      16       14.17            
      17       14.14            
      18       14.12            
      19       14.08            
      20       14.06            

Table of speed in Mb/s:
level       zstd        zlib     
    -100      990.98            
     -50      829.71            
     -20      604.65            
     -10      544.87            
      -5      486.82            
      -4      473.44            
      -3      457.86            
      -2      438.94            
      -1      421.48            
       1      371.45      122.09
       2      338.99      108.41
       3      248.63       91.23
       4      249.55       77.87
       5      149.08       60.91
       6       97.56       42.00
       7       80.16       32.16
       8       62.31       15.40
       9       45.51       13.40
      10       33.03            
      11       22.90            
      12       17.58            
      13       13.64            
      14        9.66            
      15        6.86            
      16        5.05            
      17        3.64            
      18        2.16            
      19        2.04            
      20        1.12            

Decompression

Table of speed in Mb/s:
level       zstd        zlib     
    -100     2793.54            
     -50     1918.92            
     -20     1364.04            
     -10     1312.15            
      -5     1234.27            
      -4     1206.13            
      -3     1184.73            
      -2     1172.13            
      -1     1145.40            
       1     1021.16      323.78
       2      930.12      328.26
       3      998.95      335.72
       4      902.10      329.63
       5      890.62      334.98
       6      920.96      336.79
       7      981.44      331.95
       8     1021.53      340.58
       9     1031.44      337.56
      10     1045.02            
      11     1032.52            
      12     1021.93            
      13     1026.44            
      14     1068.57            
      15     1100.77            
      16     1060.96            
      17     1047.56            
      18     1038.88            
      19     1042.81            
      20     1047.33            
sergey-dryabzhinsky commented 3 years ago

Done in #57