inikep / lzbench

lzbench is an in-memory benchmark of open-source LZ77/LZSS/LZMA compressors
866 stars 181 forks source link

lzbench reports weird results when built on ARMv7hf 32 bit (wrong compr. size numbers) #6

Closed xcrh closed 8 years ago

xcrh commented 8 years ago

Configuration: Ubuntu @ ARMv7, hardfloat ABI, 32-bit LE CPU. lzbench: current version, at commit 5937923 Compiler: GCC 4.8.1 (Linaro) - armhf abi, target is ARMv7 (Allwinner A10 based board). Flags: defaults from makefile, I've only changed the following lines to match my system better:

Problem description:

Note: Compr size looks pretty close to 2^32, which could be something like integer overflow or other bug related to dealing with (32-bit) integers. Keep in mind that on ARM and in general, sizeof(int), long, pointers, etc could vary and you can't expect it to have certain size, unless you go for something like uint32_t, etc from modern C standards.

And btw, when it comes to 64 bits, there're 64-bit ARMs and they seems to have sunrise of their popularity, so you better do not expect each 64 bit CPU to be x86. Some could be ARMs, etc.

Bonus: If you want a test run results from rather strange machine, I can launch whatever benchmark you want, subject to machine limits (only 1Gb of RAM for everything, no swap, CPU and RAM aren't exactly fastest, memcpy tops at 256Mb/sec or so).

Example output on just a 512Kb file follows:

$ ./lzbench -i15 tst 
lzbench 0.8 (32-bit Linux)   Assembled by P.Skibinski
| Compressor name             | Compression| Decompress.| Compr. size | Ratio |
| memcpy                      |   256 MB/s |   256 MB/s |  4295491584 |100.00 |
| density 0.12.5 beta level 1 |    32 MB/s |    46 MB/s |  4295404942 | 83.47 |
| density 0.12.5 beta level 2 |    12 MB/s |    15 MB/s |  4295370026 | 76.81 |
| density 0.12.5 beta level 3 |  6.65 MB/s |  6.83 MB/s |  4295348558 | 72.72 |
| fastlz 0.1 level 1          |    36 MB/s |    85 MB/s |  4295318107 | 66.91 |
| fastlz 0.1 level 2          |    30 MB/s |    85 MB/s |  4295314572 | 66.24 |
| lz4 r131                    |    32 MB/s |   128 MB/s |  4295315866 | 66.48 |
| lz4fast r131 level 3        |    51 MB/s |   170 MB/s |  4295343773 | 71.81 |
| lz4fast r131 level 17       |   128 MB/s |   256 MB/s |  4295408636 | 84.18 |
| lz5 r131b                   |  7.31 MB/s |    85 MB/s |  4295302981 | 64.03 |
| lzf 3.6 level 0             |    17 MB/s |    85 MB/s |  4295315797 | 66.47 |
| lzf 3.6 level 1             |    16 MB/s |    85 MB/s |  4295313702 | 66.07 |
| lzjb 2010                   |    32 MB/s |    73 MB/s |  4295387745 | 80.19 |
| lzo 2.09 level 1            |    19 MB/s |   102 MB/s |  4295315275 | 66.37 |
| lzo 2.09 level 1001         |    24 MB/s |   102 MB/s |  4295311518 | 65.66 |
| lzo 2.09 level 2001         |    22 MB/s |   102 MB/s |  4295310634 | 65.49 |
| lzo 2.09 level 3001         |    73 MB/s |   170 MB/s |  4295383550 | 79.39 |
| lzo 2.09 level 4001         |    73 MB/s |   170 MB/s |  4295384028 | 79.49 |
| lzrw 15-Jul-1991 level 1    |    34 MB/s |    64 MB/s |  4295351851 | 73.35 |
| lzrw 15-Jul-1991 level 2    |    32 MB/s |    73 MB/s |  4295350191 | 73.03 |
| lzrw 15-Jul-1991 level 3    |    34 MB/s |    64 MB/s |  4295330009 | 69.18 |
| lzrw 15-Jul-1991 level 4    |    34 MB/s |    39 MB/s |  4295324550 | 68.14 |
| lzrw 15-Jul-1991 level 5    |    11 MB/s |    39 MB/s |  4295314410 | 66.21 |
| pithy 2011-12-24 level 0    |    73 MB/s |   170 MB/s |  4295338432 | 70.79 |
| pithy 2011-12-24 level 3    |    46 MB/s |   170 MB/s |  4295330983 | 69.37 |
| pithy 2011-12-24 level 6    |    17 MB/s |   128 MB/s |  4295322934 | 67.83 |
| pithy 2011-12-24 level 9    |    11 MB/s |   128 MB/s |  4295319890 | 67.25 |
| quicklz 1.5.0 level 1       |    46 MB/s |    42 MB/s |  4295312758 | 65.89 |
| quicklz 1.5.0 level 2       |    17 MB/s |    34 MB/s |  4295295219 | 62.55 |
| shrinker 0.1                |    25 MB/s |   128 MB/s |  4295305322 | 64.47 |
| snappy 1.1.3                |    42 MB/s |   128 MB/s |  4295326217 | 68.46 |
| tornado 0.6a level 1        |    21 MB/s |    39 MB/s |  4295367013 | 76.24 |
| tornado 0.6a level 2        |    13 MB/s |    34 MB/s |  4295316733 | 66.65 |
| tornado 0.6a level 3        |  4.83 MB/s |    12 MB/s |  4295259130 | 55.66 |
| zstd v0.3.6                 |    14 MB/s |    36 MB/s |  4295282462 | 60.11 |
done... (15 iterations, chunk_size=512 KB, min_compr_speed=0 MB)

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/4.8/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.8.1-10ubuntu9' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libitm --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-armhf/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-armhf --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-armhf --with-arch-directory=arm --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --enable-multilib --disable-sjlj-exceptions --with-arch=armv7-a --with-fpu=vfpv3-d16 --with-float=hard --with-mode=thumb --disable-werror --enable-checking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu9) 
xcrh commented 8 years ago

NB: I've fixed issue like this:

diff --git a/_lzbench/lzbench.cpp b/_lzbench/lzbench.cpp
index d7bb8b5..522eda9 100644
--- a/_lzbench/lzbench.cpp
+++ b/_lzbench/lzbench.cpp
@@ -76,7 +76,7 @@ typedef struct string_table
 {
     std::string column1;
     float column2, column3, column5;
-    size_t column4;
+    uint64_t column4;
     string_table(std::string c1, float c2, float c3, size_t c4, float c5) : column1(c1), column2(c2), column3(c3), column4(c4), column5(c5) {}
 } string_table_t;

...not sure if printf expects exactly this, but now I'm getting proper compr. size, so it seems is was size_t on ARM to blame.

jibsen commented 8 years ago

There was no portable way to print a size_t value with printf() before C99 (%zu). Sadly MSVC is mostly stuck at C89 and doesn't support it.

xcrh commented 8 years ago

I wouldn't expect too much from Microsoft. Though failing to implement standard which exists for something like 16 years... they should try to get into Guinnes World Records, really notewothy "achievement". I wonder if they would manage to implement C11 in XXI century, or it going to be delayed until XXII?

inikep commented 8 years ago

this bug concerns all 32-bit compilations (I didn't tested 32-bit) and your fix is fine, thanks xcrh