Closed eribertomota closed 1 year ago
Hi @eribertomota!
The Debian Project has a CI system to test any package over the following architectures: amd64, armel, armhf, i386, ppc64el and s390x.
All those architectures but s390x are little-endian (well, some of them are bi-endian, though little-endian by default, but I digress), thus my first guess is that is an endianess-related problem.
Only SHA1 differ between both architectures. Executing the sha1sum command over the file on s390x will produce the right hash:
By skimming through the code, I saw that the SHA1 implementation relies on some #define
s (RUNTIME_ENDIAN
and WORDS_BIGENDIAN
) to determine processor endianess during runtime and during the build, while the other hashing implementations don't use them, as can be verified by the following grep on the project root dir:
find . -type f -exec egrep -H '(RUNTIME_ENDIAN|WORDS_BIGENDIAN)' {} \;
I found that WORDS_BIGENDIAN
should be defined by configure.ac
by using the AC_C_BIGENDIAN
macro [^1]. I couldn't find anything about the other (RUNTIME_ENDIAN
) macro, but I think we should do something like OpenLDAP did to enable determining endianess during runtime ^2.
I'll have a deeper look at it, by trying to generate the same output as s390x on amd64 by "hardcoding" the defines at strategic places for debugging. Anyway, if I eventually come to a conclusion, I won't be able to test it on s390x. But then we can make a tentative branch for you to test it there when we have something, if it is ok to you.
[^1]: https://www.gnu.org/software/autoconf/manual/autoconf-2.62/autoconf.pdf, p.75.
Hi @davidpolverari
I won't be able to test it on s390x. But then we can make a tentative branch for you to test it there when we have something, if it is ok to you.
Yes! Thanks a lot! Go ahead!
I'll have a deeper look at it, by trying to generate the same output as s390x on amd64 by "hardcoding" the defines at strategic places for debugging. Anyway, if I eventually come to a conclusion, I won't be able to test it on s390x. But then we can make a tentative branch for you to test it there when we have something, if it is ok to you.
Indeed, by making a test on amd64 with the changes below, I got dcfldd to do the opposite it should do when hashing SHA1, treating it as a big-endian arch. I think that by using the AC_C_BIGENDIAN
macro on configure.ac
, we will be able to solve the problem on s390x. Below you can find the diff for the changes I used to test and the test results after using them:
--- dcfldd-1.7.1.orig/src/sha1.c
+++ dcfldd-1.7.1/src/sha1.c
@@ -51,6 +51,8 @@
#include <string.h>
+#define WORDS_BIGENDIAN
+
#include "sha1.h"
#ifndef lint
Total (md5): 92994b0ce292a217e3e3bc31b639e565
Total (sha1): 68afccc873eb954866097c08d4ad9d9643735f2c
Total (sha256): 02fd428a4671925e4ca61541b9fac648f4ccdccad65602bfd3256ba14e59489c
Total (sha384): 53b8374607a8258de4173265bfbfa6120093fd42090a92fd589cf2c6c16b4e421b5135777714976713f7949715720a83
Total (sha512): 969a39bf47b5f12d81121084f19cb5ae250e0c0ea5b7c6d82cf08131acde8e1955d137612c2f6b255e25b0e28f96f93586f90f06965cb8f719ed7fbdd95cc8d4
I will prepare the new branch and upload it soon.
I have prepared a tentative fix (https://github.com/resurrecting-open-source-projects/dcfldd/commit/cb54a40d831999fff4158e3ac1c2c561c3dea59f) on the bugfix/issue-11
branch. Please, let me know if it worked on s390x after you try it. We can later merge it into master if it does. Thanks!
Note: maybe adding only build-time endianness configuration will not be enough, as several posts over the Internet report inconsistencies about involved compiler flags. We also might need to add runtime endianness detection support (by using the same approach as OpenLDAP). I will work on that too.
I added another commit to the bugfix/issue-11
branch that adds a configure
option (--enable-runtime-endian-check
) to enable runtime endianness checking, in addition to the build-time check. It is disabled by default.
Worked fine on s390x!
Great! Thanks for the test! I made the runtime endianness checking the default during the build. Should the user want to disable it, it can be done by using the --disable-runtime-endian-check
during ./configure
. I will merge it on master now.
The Debian Project has a CI system to test any package over the following architectures: amd64, armel, armhf, i386, ppc64el and s390x.
The following test fails on s390x:
dcfldd if=test.txt hash=md5,sha1,sha256,sha384,sha512 2>&1 | grep -C 20 bc0e4b74695142e0a0bdae87aea310d7078866cb
I attached the file test.txt to this issue. The grep shown above will test the SHA1 result. The other hashes are right. See below the result of the test over amd64 WITHOUT the grep:
Now, over s390x:
Only SHA1 differ between both architectures. Executing the sha1sum command over the file on s390x will produce the right hash:
test.txt.gz