3.2.7: test suite is failing

[kloczek]

+ /usr/bin/make -O -j48 V=1 VERBOSE=1 check
Making check in m4
make[1]: Nothing to be done for 'check'.
Making check in build
Making check in vs2008
make[2]: Nothing to be done for 'check'.
Making check in vs2010
make[2]: Nothing to be done for 'check'.
Making check in vs2017
make[2]: Nothing to be done for 'check'.
make[2]: Nothing to be done for 'check-am'.
Making check in util
Making check in .
make[2]: Nothing to be done for 'check-am'.
Making check in gmime
Making check in .
make[2]: Nothing to be done for 'check-am'.
Making check in tests
Making check in .
/usr/bin/make  check-local
make[3]: Entering directory '/home/tkloczko/rpmbuild/BUILD/gmime-3.2.7/tests'
failed_tests=""; \
failed=0; \
passed=0; \
 \
for test in test-iconv test-streams test-cat test-encodings test-filters test-headers test-mime-part test-partial test-mbox test-autocrypt test-mime test-pgp test-pgpmime; do \
        testdatadir="data/`echo ${test} | sed -e 's/test-//'`"; \
        testdir="./${testdatadir}"; \
        if [ ! -d "${testdir}" ]; then \
                testdir="${testdatadir}"; \
        fi; \
        ./${test} -v ${testdir}; \
        if [ $? -ne 0 ]; then \
                failed_tests="${failed_tests}\n ${test}"; \
                failed=`expr ${failed} + 1`; \
        else \
                passed=`expr ${passed} + 1`; \
        fi; \
done; \
 \
echo "Results: ${passed} tests passed; ${failed} tests failed."; \
if [ ${failed} -ne 0 ]; then \
        echo -e "Failed tests: ${failed_tests}"; \
        exit 255; \
fi
Testing charset conversion utils: passed
Testing Stream tests: passed
Testing GMimeStreamCat: passed
Testing Content-Transfer-Encoding: passed
Testing base64: passed
Testing uuencode: passed
Testing quoted-printable: passed
Testing GMimeFilter: passed
Testing indexing: passed
Testing removing: passed
Testing removing at an index: passed
Testing header synchronization: passed
Testing header formatting: passed
Testing GMimePart: passed
Testing message/partial: passed
Testing Mbox parser: passed
Testing Autocrypt: generate headers: passed
Testing Autocrypt: import OpenPGP secret key: passed
Testing Autocrypt: parse messages: passed
Testing Autocrypt: inject headers: passed
Testing addr-spec parser (strict): passed
Testing addr-spec parser (loose): passed
Testing date parser: passed
Testing rfc2047 encoding/decoding (strict): passed
Testing rfc2047 encoding/decoding (loose): passed
Testing rfc2184 encoding/decoding: passed
Testing quoted-strings: passed
Testing header folding: passed
Testing references: passed
Testing GnuPG crypto context: failed (3 errors, 0 warnings)
Testing PGP/MIME implementation: failed (2 errors, 0 warnings)
Results: 11 tests passed; 2 tests failed.
Failed tests:
 test-pgp
 test-pgpmime
make[3]: *** [Makefile:1141: check-local] Error 255
make[3]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/gmime-3.2.7/tests'
make[2]: *** [Makefile:978: check-am] Error 2
make[1]: *** [Makefile:827: check-recursive] Error 1
make: *** [Makefile:542: check-recursive] Error 1

[jstedfast]

Try running the tests manually like so:

./test-pgp -vvvv
./test-pgpmime -vvvv

What are the outputs of each of them?

[kloczek]

Forrgot to send ypu what you've been asking for.

Sorry

[tkloczko@barrel tests]$ ./test-pgp -vvvv
Checking GMimeGpgContext::import... PASSED
Checking GMimeGpgContext::export... PASSED
signature (pgp-sha1):
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

this is some cleartext
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQTaJtq3VyocXyqxp0u0d2DpUYLCAAUCXoI5TAAKCRC0d2DpUYLC
ALv+AJ9GA8NiYjCZU/o4daohctU6GPR6GgCeL6Y3pN8g53qE3G8DgVDKYu/VITw=
=OLGf
-----END PGP SIGNATURE-----

Checking GMimeGpgContext::sign... PASSED
GMimeGpgContext::sign failed: signature BAD
Checking GMimeGpgContext::verify... FAILED
signature (pgp-sha1):
-----BEGIN PGP SIGNATURE-----

iF0EABECAB0WIQTaJtq3VyocXyqxp0u0d2DpUYLCAAUCXoI5TQAKCRC0d2DpUYLC
AOrZAKDDpGxSH8VDRTVE1vOeFGkZwc9kqACg02f0YCI1p5MBq2GTNCT/M1l5d+o=
=zOhM
-----END PGP SIGNATURE-----

Checking GMimeGpgContext::sign (detached)... PASSED
GMimeGpgContext::sign (detached) failed: signature BAD
Checking GMimeGpgContext::verify (detached)... FAILED
ciphertext:
-----BEGIN PGP MESSAGE-----

hQIOAwTJ0obO8knpEAf+LpUdWWYZQfUe5RX9+EDPH0h/WYRZt0O8Pnf1VXbUrAzh
rocg+dnSJoWaL8X06qPtS2mfHXdIIAZXuh5yhTinR+JYJXs+xTRHRxNu8ZRqdCDT
rw6x4PXMyGSlTrJDZ9NP6FifRnb/oKOMQzdFzkAdgfRNvJ82mVNI+484oUEV0uIu
YV7i632SKW+as1ieYYVsZlgPp2nrMRBRk+rpUkj/DL0IhgPVrzGYjWxX59NEB5ku
qUfPNdv1cyAUkrWnMFtZQepg2srWdk00zVbPtizmS08c2ek+OKBmpSGdabegpHw0
FmKks1Q6V2Y+Nf95MTJryEqYAw+lFx+uu79QIfFaxAgAgukwUyTpy1ShJsRvzuCm
8JSWQV6Ysuea+nHWwE1tzfhqEseIehbVgQydfHMzfOWUZHNWERcvGpztfJT07oxB
jnZApo8+kbWw59+chg2PJY4JT/EDHpqoV2GfaS2RPgc5u1zf0BnhXeZlDHWpGJ2z
iZUvRhTo43eiaTFtrRcvb9CcbCrVD6tqux0JmA6Ux0Vv5U6pRs96U8JmnnbyN7k4
TTltlLiG6XYNkIwbqo5NIqtJDquE8vXU7aeoUMnS2gBYiI10tuvDJmpjp9ymcyDv
SzJ+dKNu+sREZbfUwFHs/Xc81vOsuUjNSm77O6VS9zPtfkHHnXaewIzvDP34wGce
JdJRAT/bbGAw+kUIMgOLeJMDiqbnmb1zPFIZWjf/kEAzg3EPD65IgwJ1T33/jO4g
7iejz3U6JRxyXkVT9fZD+f+glOXdOU+DlQXCkaYZ1Rvw8hN5
=ZEYJ
-----END PGP MESSAGE-----

Checking GMimeGpgContext::encrypt... PASSED
Checking GMimeGpgContext::decrypt... PASSED
ciphertext:
-----BEGIN PGP MESSAGE-----

hQIOAwTJ0obO8knpEAf9H7CLXyTZCvmgBwc2Zv5jQ3g2ydeATi0c55d1/WUNJAiZ
zym4oed7csEdXa12j7QPRrsjDWiAklBVl5BN/LRatPGjxnh+wUzMXidfe0ziPzzm
QkqG5yAsXTg+T8+GaIuCYw+5bn0maQA4gudwfNdgw+1G7BFGAWqTwk4me7+oP/Tl
Z8kZiNCrKYlZHeKwl06z4Q5UwEWgYuNxd7TcpAdFUBLrSrT5ogiPTgEJ8BCWGEPn
PRliCJ4M7yOKq70KToXFfSNeLuwBtjwIDd9YiIgFrXQ7LSMrTKDlBfZZIYa+BJOt
yQaG9NZ+hL+d4sCcJB1MbbYBsuh6JiIM+cpEhqI3Egf9Hg8t2yPDgOo3MV/suDTV
1OeRsK9vTuTq367lEsu9t9wdPOJCetHkmc2XF+aN0EH2wvsG+z0+qnIuV0ofuCXI
pq/j9fh+ufQf5acJ1T+gTukzCUFV5DzSV+6cV1gSeVeGZWJxW1TkbxFEE13+h8xW
IlXcI4kj1ohcNxFH3c97ZxJPaW9RrYtHJZeo0qJpXweUeC5aB3f+WaIF7ElX+TmO
OCQSE6NPcufS4nnlbtn3zD+yv6R1mwAJNqCeGpsitSReWTXAWEjSKMoFsBTO+D+i
tDfqouEvinROgqokQzyzpwQ77EGB55Pm0Gr/P0JtTWFmKoq1bJcn9QXyJTl1Kv5c
99K1AX9PFkzaAGZZp07vIXOOVI+E0j7ThWOawlCNauNP660DprBkqoKiKQKqq0rh
PnMy4D4AA6LvWL6xvEkrugwR/QvgISG+kICJ5tRGfqJeJQohRpGPy5NZoucCOMbk
bhwqFdP7rh0XbITMIRkJb3SoNXFEhpzKpKvdskruar5OPNb6lKeux5WFeZulSwEU
rH/l5BE6eFCCqZdLH13k0LCvgbxngMtG844zm01kq3giWKJ2iv1Xbg==
=6JDc
-----END PGP MESSAGE-----

Checking GMimeGpgContext::encrypt+sign... PASSED
GMimeGpgContext::encrypt+sign failed: expected GOOD signature
Checking GMimeGpgContext::decrypt+verify... FAILED
Checking GMimeFilterOpenPGP::public key block... PASSED
Checking GMimeFilterOpenPGP::private key block... PASSED
Checking GMimeFilterOpenPGP::signed message block... PASSED
Checking GMimeFilterOpenPGP::encrypted message block... PASSED
Testing GnuPG crypto context: failed (3 errors, 0 warnings)
[tkloczko@barrel tests]$ ./test-pgpmime -vvvv
Checking GMimeGpgContext::import... PASSED
Overall status: (0x4) BAD

Signatures:
        Name: no.user
        KeyId: B47760E95182C200
        UserID: no.user (no.comment) <no.user@no.domain>
        Fingerprint: B47760E95182C200
        Trust: Unknown
        Status: BAD
        Signature made on Thu Jan  1 01:00:00 1970
        Signature never expires
        Errors: None
multipart/signed failed: signature status was BAD
Checking multipart/signed... FAILED
Checking multipart/encrypted... PASSED
Overall status: (0x4) BAD

Signatures:
        Name: no.user
        KeyId: B47760E95182C200
        UserID: no.user (no.comment) <no.user@no.domain>
        Fingerprint: B47760E95182C200
        Trust: Unknown
        Status: BAD
        Signature made on Thu Jan  1 01:00:00 1970
        Signature never expires
        Errors: None
multipart/encrypted+sign failed: signature status expected to be GOOD
Checking multipart/encrypted+sign... FAILED
Checking rfc4880 sign... PASSED
Checking rfc4880 encrypt... PASSED
Checking rfc4880 sign+encrypt... PASSED
Testing PGP/MIME implementation: failed (2 errors, 0 warnings)

[jstedfast]

What system is this failing on? I can't reproduce these failures. I also haven't changed anything that would cause this, so I'm tempted to just close it anyway.

[kloczek]

It is my own system on which I've started working +1 year ago with almost entirely rebuild Fedora where all packages are rebuild using LTO and --as-needed linker option.

[jstedfast]

I'm not sure what the problem could be. GpgME is used to generate the signature and verify it, so if it's failing on your system, I imagine it has to be specific to your system because I can't make it fail on my system.

[kloczek]

OK. How could you approach to diagnose those faults?

Just please give me some hints and I'll try to follow :)

[jstedfast]

I guess your best bet is to git-bisect to figure out which commit broke it and go from there.

[kloczek]

Just tested 3.2.12 and test suite still fails

```console + cd gmime-3.2.12 + /usr/bin/make -O -j48 V=1 VERBOSE=1 check Making check in m4 make[1]: Nothing to be done for 'check'. Making check in build Making check in vs2008 make[2]: Nothing to be done for 'check'. Making check in vs2010 make[2]: Nothing to be done for 'check'. Making check in vs2017 make[2]: Nothing to be done for 'check'. make[2]: Nothing to be done for 'check-am'. Making check in util Making check in . make[2]: Nothing to be done for 'check-am'. Making check in gmime Making check in . make[2]: Nothing to be done for 'check-am'. Making check in tests Making check in . /usr/bin/make check-local make[3]: Entering directory '/home/tkloczko/rpmbuild/BUILD/gmime-3.2.12/tests' failed_tests=""; \ failed=0; \ passed=0; \ \ for test in test-iconv test-streams test-cat test-encodings test-filters test-headers test-mime-part test-partial test-mbox test-autocrypt test-mime ; do \ testdatadir="data/`echo ${test} | sed -e 's/test-//'`"; \ testdir="./${testdatadir}"; \ if [ ! -d "${testdir}" ]; then \ testdir="${testdatadir}"; \ fi; \ ./${test} -v ${testdir}; \ if [ $? -ne 0 ]; then \ failed_tests="${failed_tests}\n ${test}"; \ failed=`expr ${failed} + 1`; \ else \ passed=`expr ${passed} + 1`; \ fi; \ done; \ \ echo "Results: ${passed} tests passed; ${failed} tests failed."; \ if [ ${failed} -ne 0 ]; then \ echo -e "Failed tests: ${failed_tests}"; \ exit 255; \ fi Testing charset conversion utils: passed Testing Stream tests: passed Testing GMimeStreamCat: passed Testing Content-Transfer-Encoding: passed Testing base64: passed Testing uuencode: passed Testing quoted-printable: passed Testing GMimeFilter: failed (1 errors, 0 warnings) Testing indexing: passed Testing removing: passed Testing removing at an index: passed Testing header synchronization: passed Testing header formatting: passed Testing GMimePart: passed Testing message/partial: passed Testing Mbox parser: passed Testing Autocrypt: generate headers: passed Testing Autocrypt: parse messages: passed Testing addr-spec parser (strict): passed Testing addr-spec parser (loose): passed Testing date parser: passed Testing rfc2047 encoding/decoding (strict): passed Testing rfc2047 encoding/decoding (loose): passed Testing rfc2184 encoding/decoding: passed Testing quoted-strings: passed Testing header folding: passed Testing references: passed Results: 10 tests passed; 1 tests failed. Failed tests: test-filters make[3]: *** [Makefile:1144: check-local] Error 255 make[3]: Leaving directory '/home/tkloczko/rpmbuild/BUILD/gmime-3.2.12/tests' make[2]: *** [Makefile:981: check-am] Error 2 make[1]: *** [Makefile:831: check-recursive] Error 1 make: *** [Makefile:543: check-recursive] Error 1 ```

[jstedfast]

Run ./test-filters -vvvv

[kloczek]

Here it is

[tkloczko@devel-g2v tests]$ ./test-filters -vvvv
Checking GMimeFilterCharset (cyrillic utf-8 -> cp1251)... PASSED
Checking GMimeFilterCharset (cyrillic cp1251 -> utf-8)... PASSED
Checking GMimeFilterCharset (cyrillic utf-8 -> iso-8859-5)... PASSED
Checking GMimeFilterCharset (cyrillic iso-8859-5 -> utf-8)... PASSED
Checking GMimeFilterCharset (cyrillic utf-8 -> koi8-r)... PASSED
Checking GMimeFilterCharset (cyrillic koi8-r -> utf-8)... PASSED
Checking GMimeFilterCharset (japanese utf-8 -> iso-2022-jp)... PASSED
Checking GMimeFilterCharset (japanese iso-2022-jp -> utf-8)... PASSED
Checking GMimeFilterCharset (japanese utf-8 -> shift-jis)... PASSED
Checking GMimeFilterCharset (japanese shift-jis -> utf-8)... PASSED
Checking GMimeFilterEnriched (enriched.txt)... PASSED
GMimeFilterGzip::zip failed: streams are not the same length: 1212
Checking GMimeFilterGzip::zip... FAILED
Checking GMimeFilterGzip::unzip... PASSED
Checking GMimeFilterHtml (html-input.txt blockquote)... PASSED
Checking GMimeFilterHtml (html-input.txt mark)... PASSED
Checking GMimeFilterHtml (html-input.txt cite)... PASSED
Checking GMimeFilterSmtpData... PASSED
Checking GMimeFilterWindows... PASSED
Testing GMimeFilter: failed (1 errors, 0 warnings)

Just in case I'm using zlib-ng https://github.com/zlib-ng/zlib-ng/