search

jackyf / cupt

high-level package manager for Debian, inspired from APT
36 stars 4 forks source link

Failing test `query/repo-signatures/validation-errors:"empty signature"` in a Debian stretch container #7

Closed urp closed 5 years ago

urp commented 5 years ago

When trying to build the debian package in a debian stretch docker container two of the tests fail:

   #   Failed test 'validation failed'
    #   at tt/query/repo-signatures/validation-errors.t line 15.
    #                   'Package: p
    # Version: 1
    # Status: not installed
    # Source: p
    # Priority: extra
    # Uncompressed size: 0B
    # Architecture: all
    # URI: copy://./localrepo//
    # SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
    # 
    # '
    #           matches '(?^:Package)'

    #   Failed test 'error message is right'
    #   at tt/query/repo-signatures/validation-errors.t line 16.
    #                   'Package: p
    # Version: 1
    # Status: not installed
    # Source: p
    # Priority: extra
    # Uncompressed size: 0B
    # Architecture: all
    # URI: copy://./localrepo//
    # SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
    # 
    # '
    #     doesn't match '(?^m:empty signature)'
    # Looks like you failed 2 tests of 2.

#   Failed test 'empty signature'
#   at tt/query/repo-signatures/validation-errors.t line 18.
# Looks like you failed 1 test of 9.
[08:47:47] tt/query/repo-signatures/validation-errors.t ................................................... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests

This happens on master and the current release

jackyf commented 5 years ago

Thank you for the report.

Two messages you indicated are subtest names, the top-level test name is "empty signature" -> retitled the issue accordingly.

I cannot reproduce the issue using gpgv either from Debian testing (buster) or Debian stable (stretch). Moreover, the way the test failed, it's unexpected that none of other tests failed.

Questions:

  1. Were there any failures from the test suite tt/query/repo-signatures/yes-no.t?
  2. Is the failure reproducible? That is, if you repeat the build 1-2 times more, will the test(s) fail in exactly the same way?
  3. What version of gpgv Debian package the container has?
  4. What is/was the system date in the container?
urp commented 5 years ago

Thank you for answering!

  1. No other tests failed in the entire test run
  2. Yes, the problem occurs reproducibly 
  3. # gpgv --version
gpgv (GnuPG) 2.1.18
libgcrypt 1.7.6-beta
  4. date returns the correct time when run in the container
urp commented 5 years ago

I was expecting this to be related to my docker image setup but building cupt and running its tests in a plain debian container after installing source package dependencies produces the same error. Is there a specific dependency for those tests not listed in the control file? I used the following command sequence to build and run the tests 

cmake ../cupt
make
make test
jackyf commented 5 years ago

Is there a specific dependency for those tests not listed in the control file?

Unlikely. Had it been the case, a lot more tests would fail, and in a different way, too.

To find the root case, I'd like to ask you to run the specific test(s) with more debugging information. Please run the following from the build directory (everything is on the same line) and post the output: PROVE_FILTER=query/repo-signatures PROVE_BINARY="$(pwd)/cpp/console/cupt -o debug::gpgv=1" make test

urp commented 5 years ago

Output of PROVE_FILTER=query/repo-signatures PROVE_BINARY="$(pwd)/cpp/console/cupt -o debug::gpgv=1" make test:

[ 80%] Built target libcupt
[100%] Built target cupt.bin
Scanning dependencies of target test
[13:34:38] tt/query/repo-signatures/validation-errors.t .. 1/9     
    #   Failed test 'validation failed'
    #   at tt/query/repo-signatures/validation-errors.t line 15.
    #                   'D: verifying file '/cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release'
    # D: signature file is '/cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release.gpg'
    # D: keyring file is '/cupt-build/test/env/etc/apt/trusted.gpg'
    # D: gpgv command is 'gpgv --status-fd 1  --keyring /cupt-build/test/env/etc/apt/trusted.gpg /cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release.gpg /cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release 2>/dev/null || true'
    # D: fetched '[GNUPG:] NEWSIG' from gpg pipe
    # D: fetched '[GNUPG:] KEY_CONSIDERED C4D480AE234066C6489B1242BF217D59A868F816 0' from gpg pipe
    # D: fetched '[GNUPG:] SIG_ID CL+bYts0XBvAEkKKAoDfharVQao 2019-05-28 1559050478' from gpg pipe
    # D: fetched '[GNUPG:] KEY_CONSIDERED C4D480AE234066C6489B1242BF217D59A868F816 0' from gpg pipe
    # D: fetched '[GNUPG:] GOODSIG BF217D59A868F816 Cupt Test Suite <cupt-test-suite@localhost>' from gpg pipe
    # D: the verify result is 1
    # Package: p
    # Version: 1
    # Status: not installed
    # Source: p
    # Priority: extra
    # Uncompressed size: 0B
    # Architecture: all
    # URI: copy://./localrepo//
    # SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
    # 
    # '
    #           matches '(?^:Package)'

    #   Failed test 'error message is right'
    #   at tt/query/repo-signatures/validation-errors.t line 16.
    #                   'D: verifying file '/cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release'
    # D: signature file is '/cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release.gpg'
    # D: keyring file is '/cupt-build/test/env/etc/apt/trusted.gpg'
    # D: gpgv command is 'gpgv --status-fd 1  --keyring /cupt-build/test/env/etc/apt/trusted.gpg /cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release.gpg /cupt-build/test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_Release 2>/dev/null || true'
    # D: fetched '[GNUPG:] NEWSIG' from gpg pipe
    # D: fetched '[GNUPG:] KEY_CONSIDERED C4D480AE234066C6489B1242BF217D59A868F816 0' from gpg pipe
    # D: fetched '[GNUPG:] SIG_ID CL+bYts0XBvAEkKKAoDfharVQao 2019-05-28 1559050478' from gpg pipe
    # D: fetched '[GNUPG:] KEY_CONSIDERED C4D480AE234066C6489B1242BF217D59A868F816 0' from gpg pipe
    # D: fetched '[GNUPG:] GOODSIG BF217D59A868F816 Cupt Test Suite <cupt-test-suite@localhost>' from gpg pipe
    # D: the verify result is 1
    # Package: p
    # Version: 1
    # Status: not installed
    # Source: p
    # Priority: extra
    # Uncompressed size: 0B
    # Architecture: all
    # URI: copy://./localrepo//
    # SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
    # 
    # '
    #     doesn't match '(?^m:empty signature)'
    # Looks like you failed 2 tests of 2.

#   Failed test 'empty signature'
#   at tt/query/repo-signatures/validation-errors.t line 18.
# Looks like you failed 1 test of 9.
[13:34:38] tt/query/repo-signatures/validation-errors.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/9 subtests 
[13:34:38] tt/query/repo-signatures/yes-no.t ............. ok     2554 ms ( 0.02 usr  0.00 sys +  1.33 cusr  0.73 csys =  2.08 CPU)
[13:34:41]

Test Summary Report
-------------------
tt/query/repo-signatures/validation-errors.t (Wstat: 256 Tests: 9 Failed: 1)
  Failed test:  8
  Non-zero exit status: 1
Files=2, Tests=64,  3 wallclock secs ( 0.05 usr  0.01 sys +  1.60 cusr  0.83 csys =  2.49 CPU)
Result: FAIL
test/CMakeFiles/test.dir/build.make:57: recipe for target 'test/CMakeFiles/test' failed
make[3]: *** [test/CMakeFiles/test] Error 1
CMakeFiles/Makefile2:634: recipe for target 'test/CMakeFiles/test.dir/all' failed
make[2]: *** [test/CMakeFiles/test.dir/all] Error 2
CMakeFiles/Makefile2:641: recipe for target 'test/CMakeFiles/test.dir/rule' failed
make[1]: *** [test/CMakeFiles/test.dir/rule] Error 2
Makefile:292: recipe for target 'test' failed
make: *** [test] Error 2
jackyf commented 5 years ago

Thank you. It confirms that the hidden problem is in the test case setup.

The test case in question tries to remove read permissions from one of the files it just created. On your machine, that presumably fails. Unfortunately, the error checking is lacking and the subtest title is not helpful enough - added that to my TODO list.

That doesn't yet explain why it fails on your machine. In the build directory, could you run the command chmod 0220 test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_main_binary-z128_Packages? I'd now expect that to fail and tell why.

urp commented 5 years ago 
root@26282175b46b:/cupt-build# chmod 0220 test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_main_binary-z128_Packages

Result

chmod: cannot access 'test/env/var/lib/cupt/lists/copy___._localrepo_dists_testing_main_binary-z128_Packages': No such file or directory

It seems the lists subdirectory has not been created.

urp commented 5 years ago 
root@26282175b46b:/cupt-build# ls -l test/env/var/lib/    
total 12
drwxr-xr-x 2 root root 4096 Jun  2 19:28 apt
drwxr-xr-x 2 root root 4096 Jun  2 19:28 cupt
drwxr-xr-x 2 root root 4096 Jun  2 19:28 dpkg
jackyf commented 5 years ago

I see - the file was apparently removed by subsequent tests and therefore the output is inconslusive.

I've now updated master-branch of Cupt with error checks for chmod. Could you fetch it, and then run the following command in the build directory:

PROVE_FILTER=query/repo-signatures make test

jackyf commented 5 years ago

Some time passed without a follow-up information. I'm closing this bug as un-reproducible for now, please re-open it if you have an interest in debugging this further.

urp commented 5 years ago

The problem was simply that I was building cupt as root in the docker container which can not remove read permissions for root. As a workaround I created another user and used it for the build.

jackyf commented 5 years ago

I see, that explains it. Thanks for confirmation.