powturbo / TurboPFor-Integer-Compression

Fastest Integer Compression
GNU General Public License v2.0
770 stars 112 forks source link

icapp I and J arguments #113

Closed ordinaryorange closed 11 months ago

ordinaryorange commented 11 months ago

I'm unclear on the I and J arguments to icapp.

  1. What is a de/compression iteration vs a de/compression run ?
  2. Why are the I0 and J0 E and D numbers so different ?

example runs

C:\Temp\TurboPF>icapp.exe -Ft -K0 -I0 -J0 -e1-5 c:\temp\foo.csv
      size   ratio     E MB/s   D MB/s   function integer size=32 bits (lz=lz4,1) unsorted -1
       707   8.69%       6.16   598.53   1:p4nenc32         TurboPFor                   c:\temp\foo.csv
       707   8.69%       7.28  1197.06   2:p4nenc128v32     TurboPFor128                c:\temp\foo.csv
       754   9.26%      19.66   935.63   3:p4nenc256v32     TurboPFor256                c:\temp\foo.csv
       934  11.47%      18.87   426.18   4:p4ndenc32        TurboPFor    delta          c:\temp\foo.csv
       934  11.47%       6.25   311.88   5:p4ndenc128v32    TurboPFor128 delta          c:\temp\foo.csv
Best methods =1,2,3,4,5,

C:\Temp\TurboPF>icapp.exe -Ft -K0 -I1 -J1 -e1-5 c:\temp\foo.csv
      size   ratio     E MB/s   D MB/s   function integer size=32 bits (lz=lz4,1) unsorted -1
       707   8.69%    1652.68 13010.15   1:p4nenc32         TurboPFor                   c:\temp\foo.csv
       707   8.69%    1744.16 45966.10   2:p4nenc128v32     TurboPFor128                c:\temp\foo.csv
       754   9.26%    1732.81 37469.74   3:p4nenc256v32     TurboPFor256                c:\temp\foo.csv
       934  11.47%    1124.11  7542.16   4:p4ndenc32        TurboPFor    delta          c:\temp\foo.csv
       934  11.47%     411.18  3470.98   5:p4ndenc128v32    TurboPFor128 delta          c:\temp\foo.csv
Best methods =1,2,3,4,5,

C:\Temp\TurboPF>icapp.exe -Ft -K0 -I2 -J2 -e1-5 c:\temp\foo.csv
      size   ratio     E MB/s   D MB/s   function integer size=32 bits (lz=lz4,1) unsorted -1
       707   8.69%    1684.46 12759.02   1:p4nenc32         TurboPFor                   c:\temp\foo.csv
       707   8.69%    1784.50 52989.77   2:p4nenc128v32     TurboPFor128                c:\temp\foo.csv
       754   9.26%     796.14 15294.08   3:p4nenc256v32     TurboPFor256                c:\temp\foo.csv
       934  11.47%     405.67  2759.19   4:p4ndenc32        TurboPFor    delta          c:\temp\foo.csv
       934  11.47%     424.83  3708.04   5:p4ndenc128v32    TurboPFor128 delta          c:\temp\foo.csv
Best methods =1,2,3,4,5,
powturbo commented 11 months ago

A run is X iterations running within 2 seconds. A function is called X times until reaching 2 seconds running time.

0 : (I0 and J0) is used for testing only and is not accurate 3: default (same as without the options -I# and -J#) This is good enough if your cpu doesn't make throttling 15: accurate timings but can be too slow