search

freebsd / kyua

Testing framework for infrastructure software
BSD 3-Clause "New" or "Revised" License
147 stars 42 forks source link

Test failures (and how single tests can be run/debugged?) #223

Closed Apteryks closed 7 months ago

Apteryks commented 7 months ago

Hi,

I'm trying to debug the following failures:

1428/1432 passed (4 failed)
===> Execution context
Current directory: /tmp/guix-build-kyua-0.13.drv-0/source
Environment variables:
    ACLOCAL_PATH=/gnu/store/yl13jgfnp3mqh5xym4j59j0ii3gzx1gh-automake-1.16.5/share/aclocal:/gnu/store/d4ag9ngkd4yk9rqh9q60l4hwwv5zwh9h-pkg-config-0.29.2/share/aclocal:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/share/aclocal
    CPLUS_INCLUDE_PATH=/gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/include:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/include:/gnu/store/mkzjsbbsm50nm154233pmf1gvniq64c1-lutok-0.4/include:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/include:/gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/include:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/include:/gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/include:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/include:/gnu/store/x07bk294cf7xbz9wv55v8wg5l9yfr12c-make-4.4.1/include:/gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/include:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/include/c++:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/include:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/include:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/include:/gnu/store/svghsdn1x3fyili8ij791nn25zba9bpf-linux-libre-headers-5.15.49/include
    C_INCLUDE_PATH=/gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/include:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/include:/gnu/store/mkzjsbbsm50nm154233pmf1gvniq64c1-lutok-0.4/include:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/include:/gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/include:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/include:/gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/include:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/include:/gnu/store/x07bk294cf7xbz9wv55v8wg5l9yfr12c-make-4.4.1/include:/gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/include:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/include:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/include:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/include:/gnu/store/svghsdn1x3fyili8ij791nn25zba9bpf-linux-libre-headers-5.15.49/include
    GUIX_LD_WRAPPER_ALLOW_IMPURITIES=no
    GUIX_LOCPATH=/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/lib/locale
    HOME=/tmp
    KYUA_CONFDIR=/non-existent
    KYUA_DOCDIR=/tmp/guix-build-kyua-0.13.drv-0/source
    KYUA_EXAMPLESDIR=/tmp/guix-build-kyua-0.13.drv-0/source/examples
    KYUA_MISCDIR=/tmp/guix-build-kyua-0.13.drv-0/source/misc
    KYUA_STOREDIR=/tmp/guix-build-kyua-0.13.drv-0/source/store
    KYUA_STORETESTDATADIR=/tmp/guix-build-kyua-0.13.drv-0/source/store
    LC_ALL=C.UTF-8
    LIBRARY_PATH=/gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/lib:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/lib:/gnu/store/mkzjsbbsm50nm154233pmf1gvniq64c1-lutok-0.4/lib:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/lib:/gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/lib:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/lib:/gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/lib:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/lib:/gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/lib:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/lib:/gnu/store/bzmr95rvric0nyfcp9pdzw57vxz4piii-glibc-2.38-static/lib:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/lib
    MAKEFLAGS= -j4 --jobserver-auth=fifo:/tmp/guix-build-kyua-0.13.drv-0/GMfifo6741
    MAKELEVEL=2
    MFLAGS=-j4 --jobserver-auth=fifo:/tmp/guix-build-kyua-0.13.drv-0/GMfifo6741
    NIX_BUILD_CORES=0
    NIX_BUILD_TOP=/tmp/guix-build-kyua-0.13.drv-0
    NIX_STORE=/gnu/store
    OBJCPLUS_INCLUDE_PATH=/gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/include:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/include:/gnu/store/mkzjsbbsm50nm154233pmf1gvniq64c1-lutok-0.4/include:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/include:/gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/include:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/include:/gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/include:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/include:/gnu/store/x07bk294cf7xbz9wv55v8wg5l9yfr12c-make-4.4.1/include:/gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/include:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/include/c++:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/include:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/include:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/include:/gnu/store/svghsdn1x3fyili8ij791nn25zba9bpf-linux-libre-headers-5.15.49/include
    OBJC_INCLUDE_PATH=/gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/include:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/include:/gnu/store/mkzjsbbsm50nm154233pmf1gvniq64c1-lutok-0.4/include:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/include:/gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/include:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/include:/gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/include:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/include:/gnu/store/x07bk294cf7xbz9wv55v8wg5l9yfr12c-make-4.4.1/include:/gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/include:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/include:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/include:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/include:/gnu/store/svghsdn1x3fyili8ij791nn25zba9bpf-linux-libre-headers-5.15.49/include
    PATH=/tmp/guix-build-kyua-0.13.drv-0/source:/gnu/store/ks69xx3h57iidibw58jwr181gapnl4kq-autoconf-2.69/bin:/gnu/store/yl13jgfnp3mqh5xym4j59j0ii3gzx1gh-automake-1.16.5/bin:/gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/bin:/gnu/store/d4ag9ngkd4yk9rqh9q60l4hwwv5zwh9h-pkg-config-0.29.2/bin:/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/bin:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/bin:/gnu/store/rxi9rs0rnx4mhkw9ack48f46q8na5zav-tar-1.34/bin:/gnu/store/5qz9bcnhk49lphrizmrd09h7hkdf6a55-gzip-1.13/bin:/gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/bin:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/bin:/gnu/store/l7wfbns93jgymisbiawv933xswd8g5kl-diffutils-3.10/bin:/gnu/store/bxx77c79c16yxzrkl8c8pllzfrx6qdz1-patch-2.7.6/bin:/gnu/store/1vsnpgnnp9f26ni2bsj23qv1mm40zh2j-findutils-4.9.0/bin:/gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/bin:/gnu/store/gzv2myb6fnm3lqfb5g5p3c8h6n395rk9-zstd-1.5.2/bin:/gnu/store/w8jl47q83647kgwwr7s5p6na6q7xhhvw-sed-4.8/bin:/gnu/store/9fmw9ncjkky1ra61ql8002l4y5f9ggha-grep-3.11/bin:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/bin:/gnu/store/1jk5in7363nv3a5svd0y0aqzb6r3kr4m-coreutils-9.1/bin:/gnu/store/x07bk294cf7xbz9wv55v8wg5l9yfr12c-make-4.4.1/bin:/gnu/store/p7zqfnpxkqj7im10x2ncpm9l62g3zmnp-bash-minimal-5.1.16/bin:/gnu/store/mq3zapvkjg80ln8fz7g2bjx902gmr9n8-ld-wrapper-0/bin:/gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/bin:/gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/bin:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/bin:/gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/sbin:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/bin
    PKG_CONFIG_PATH=/gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/lib/pkgconfig:/gnu/store/mkzjsbbsm50nm154233pmf1gvniq64c1-lutok-0.4/lib/pkgconfig:/gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/lib/pkgconfig:/gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/lib/pkgconfig:/gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/lib/pkgconfig:/gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/lib/pkgconfig
    PWD=/tmp/guix-build-kyua-0.13.drv-0/source
    SHLVL=2
    SOURCE_DATE_EPOCH=1
    TEMP=/tmp/guix-build-kyua-0.13.drv-0
    TEMPDIR=/tmp/guix-build-kyua-0.13.drv-0
    TMP=/tmp/guix-build-kyua-0.13.drv-0
    TMPDIR=/tmp/guix-build-kyua-0.13.drv-0
    _=/gnu/store/1jk5in7363nv3a5svd0y0aqzb6r3kr4m-coreutils-9.1/bin/env
    out=/gnu/store/sg55y5agxxnb01p74h9anjsijxyd8bnf-kyua-0.13
===> integration/cmd_report_junit_test:default_behavior__ok
Result:     failed: atf-check failed; see the output of the test for details
Start time: 2024-01-29T15:46:40.260170Z
End time:   2024-01-29T15:46:40.351388Z
Duration:   0.091s

Metadata:
    allowed_architectures is empty
    allowed_platforms is empty
    description is empty
    has_cleanup = false
    is_exclusive = false
    required_configs is empty
    required_disk_space = 0
    required_files is empty
    required_memory = 0
    required_programs = kyua
    required_user is empty
    timeout = 300

Standard output:
Executing command [ env MOCK=mock1
this should not be seen
mock1 new line kyua test ]
Executing command [ ! -c kyua report-junit | awk "
BEGIN { skip = 0; }

/<\/properties>/ {
    print \"</properties>\";
    skip = 0;
    next;
}

/<properties>/ {
    print \"<properties>\";
    print \"CONTENTS STRIPPED BY TEST\";
    skip = 1;
    next;
}

{ if (!skip) print; }" ]

Standard error:
Fail: incorrect exit status: 127, expected: 0
stdout:

stderr:
execvp(!) failed: No such file or directory

Files left in work directory after failure: .kyua, expout, kyua, kyua.tmpout, unused_dbfile_name
===> integration/cmd_report_junit_test:output__explicit
Result:     failed: atf-check failed; see the output of the test for details
Start time: 2024-01-29T15:46:40.353778Z
End time:   2024-01-29T15:46:40.434983Z
Duration:   0.081s

Metadata:
    allowed_architectures is empty
    allowed_platforms is empty
    description is empty
    has_cleanup = false
    is_exclusive = false
    required_configs is empty
    required_disk_space = 0
    required_files is empty
    required_memory = 0
    required_programs = kyua
    required_user is empty
    timeout = 300

Standard output:
Executing command [ env MOCK=unused_mock kyua test ]
Executing command [ ! -c kyua report-junit --output=/dev/stdout | awk "
BEGIN { skip = 0; }

/<\/properties>/ {
    print \"</properties>\";
    skip = 0;
    next;
}

/<properties>/ {
    print \"<properties>\";
    print \"CONTENTS STRIPPED BY TEST\";
    skip = 1;
    next;
}

{ if (!skip) print; }" | sed -E \
    -e "s,( |\[|\")[0-9][0-9]*.[0-9][0-9][0-9](s]|s|\"),\1S.UUU\2,g" \
    -e "s,[0-9]{8}-[0-9]{6}-[0-9]{6},YYYYMMDD-HHMMSS-ssssss,g" \
    -e "s,[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}Z,YYYY-MM-DDTHH:MM:SS.ssssssZ,g" ]

Standard error:
Fail: incorrect exit status: 127, expected: 0
stdout:

stderr:
execvp(!) failed: No such file or directory

Files left in work directory after failure: .kyua, report, unused_dbfile_name
===> integration/cmd_report_test:output__explicit
Result:     failed: atf-check failed; see the output of the test for details
Start time: 2024-01-29T15:46:41.415766Z
End time:   2024-01-29T15:46:41.511382Z
Duration:   0.096s

Metadata:
    allowed_architectures is empty
    allowed_platforms is empty
    description is empty
    has_cleanup = false
    is_exclusive = false
    required_configs is empty
    required_disk_space = 0
    required_files is empty
    required_memory = 0
    required_programs = kyua
    required_user is empty
    timeout = 300

Standard output:
Executing command [ env MOCK=unused_mock _=fake-value kyua test ]
Executing command [ ! -c kyua report --output=/dev/stdout | sed -E \
    -e "s,( |\[|\")[0-9][0-9]*.[0-9][0-9][0-9](s]|s|\"),\1S.UUU\2,g" \
    -e "s,[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}Z,YYYY-MM-DDTHH:MM:SS.ssssssZ,g" ]

Standard error:
Fail: incorrect exit status: 127, expected: 0
stdout:

stderr:
execvp(!) failed: No such file or directory

Files left in work directory after failure: .kyua, dbfile_name, report
===> integration/cmd_report_test:verbose
Result:     failed: atf-check failed; see the output of the test for details
Start time: 2024-01-29T15:46:42.199039Z
End time:   2024-01-29T15:46:42.299681Z
Duration:   0.101s

Metadata:
    allowed_architectures is empty
    allowed_platforms is empty
    description is empty
    has_cleanup = false
    is_exclusive = false
    required_configs is empty
    required_disk_space = 0
    required_files is empty
    required_memory = 0
    required_programs = kyua
    required_user is empty
    timeout = 300

Standard output:
Executing command [ env MOCK=mock1
has multiple lines
and terminates here _=fake-value kyua test ]
Executing command [ ! -c kyua report --verbose | sed -E \
    -e "s,( |\[|\")[0-9][0-9]*.[0-9][0-9][0-9](s]|s|\"),\1S.UUU\2,g" \
    -e "s,[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{6}Z,YYYY-MM-DDTHH:MM:SS.ssssssZ,g" ]

Standard error:
Fail: incorrect exit status: 127, expected: 0
stdout:

stderr:
execvp(!) failed: No such file or directory

Files left in work directory after failure: .kyua, dbfile_name, expout
===> Failed tests
integration/cmd_report_junit_test:default_behavior__ok  ->  failed: atf-check failed; see the output of the test for details  [0.091s]
integration/cmd_report_junit_test:output__explicit  ->  failed: atf-check failed; see the output of the test for details  [0.081s]
integration/cmd_report_test:output__explicit  ->  failed: atf-check failed; see the output of the test for details  [0.096s]
integration/cmd_report_test:verbose  ->  failed: atf-check failed; see the output of the test for details  [0.101s]
===> Summary
Results read from /tmp/.kyua/store/results.tmp_guix-build-kyua-0.13.drv-0_source.20240129-154618-396950.db
Test cases: 1432 total, 18 skipped, 0 expected failures, 0 broken, 4 failed
Start time: 2024-01-29T15:46:18.436430Z
End time:   2024-01-29T15:47:16.282536Z
Total time: 52.356s
make[1]: *** [Makefile:10873: check-kyua] Error 1
make[1]: Leaving directory '/tmp/guix-build-kyua-0.13.drv-0/source'
make: *** [Makefile:9983: check-am] Error 2

Test suite failed, dumping logs.

--- ./bootstrap/testsuite.log -----------------------------------------------

## ------------------------------------------ ##
## Kyua 0.13 test suite: bootstrapping tests. ##
## ------------------------------------------ ##

testsuite: command line was:
  $ ./testsuite 

## --------- ##
## Platform. ##
## --------- ##

hostname = localhost
uname -m = x86_64
uname -r = 5.15.131-gnu
uname -s = Linux
uname -v = #1 SMP 1

/usr/bin/uname -p = unknown
/bin/uname -X     = unknown

/bin/arch              = unknown
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /tmp/guix-build-kyua-0.13.drv-0/source/bootstrap
PATH: /tmp/guix-build-kyua-0.13.drv-0/source
PATH: /gnu/store/ks69xx3h57iidibw58jwr181gapnl4kq-autoconf-2.69/bin
PATH: /gnu/store/yl13jgfnp3mqh5xym4j59j0ii3gzx1gh-automake-1.16.5/bin
PATH: /gnu/store/46gzlnqz2000hsysmcz6pnz480f0vjrp-gdb-12.1/bin
PATH: /gnu/store/d4ag9ngkd4yk9rqh9q60l4hwwv5zwh9h-pkg-config-0.29.2/bin
PATH: /gnu/store/nllv9l08hgjsqp81ccxgzmpfmgz3g8g5-atf-0.21/bin
PATH: /gnu/store/1igw9i48is5nhc0bcax4xi9akhn2542p-sqlite-3.39.3/bin
PATH: /gnu/store/rxi9rs0rnx4mhkw9ack48f46q8na5zav-tar-1.34/bin
PATH: /gnu/store/5qz9bcnhk49lphrizmrd09h7hkdf6a55-gzip-1.13/bin
PATH: /gnu/store/d1h84v83fp1ciby8crnr2ivy0r18ll7d-bzip2-1.0.8/bin
PATH: /gnu/store/9492zd1nclx530rrzcfzip88i1i81q4d-file-5.45/bin
PATH: /gnu/store/l7wfbns93jgymisbiawv933xswd8g5kl-diffutils-3.10/bin
PATH: /gnu/store/bxx77c79c16yxzrkl8c8pllzfrx6qdz1-patch-2.7.6/bin
PATH: /gnu/store/1vsnpgnnp9f26ni2bsj23qv1mm40zh2j-findutils-4.9.0/bin
PATH: /gnu/store/zchl0dbq85g65vcaw85gqpchczllkj9l-gawk-5.3.0/bin
PATH: /gnu/store/gzv2myb6fnm3lqfb5g5p3c8h6n395rk9-zstd-1.5.2/bin
PATH: /gnu/store/w8jl47q83647kgwwr7s5p6na6q7xhhvw-sed-4.8/bin
PATH: /gnu/store/9fmw9ncjkky1ra61ql8002l4y5f9ggha-grep-3.11/bin
PATH: /gnu/store/x7x2j8racm7fqlbxysqx80r5bhxwx09d-xz-5.4.5/bin
PATH: /gnu/store/1jk5in7363nv3a5svd0y0aqzb6r3kr4m-coreutils-9.1/bin
PATH: /gnu/store/x07bk294cf7xbz9wv55v8wg5l9yfr12c-make-4.4.1/bin
PATH: /gnu/store/p7zqfnpxkqj7im10x2ncpm9l62g3zmnp-bash-minimal-5.1.16/bin
PATH: /gnu/store/mq3zapvkjg80ln8fz7g2bjx902gmr9n8-ld-wrapper-0/bin
PATH: /gnu/store/z230vk3p8jbc17hn2rgyrhlfzf4x0gvb-binutils-2.41/bin
PATH: /gnu/store/8yz8wfnzz3m4lmzr9hwmk9w9iiplnd8c-gcc-11.4.0/bin
PATH: /gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/bin
PATH: /gnu/store/bspzc51j5qk885d3hn472z7zhvhvv52i-glibc-2.38/sbin
PATH: /gnu/store/nqh4949kf38hky9jh9y4y4j8nbziprk7-lua-5.2.4/bin

testsuite: atconfig:
| # Configurable variable values for building test suites.
| # Generated by ./config.status.
| # Copyright (C) 2012 Free Software Foundation, Inc.
| 
| # The test suite will define top_srcdir=/../.. etc.
| at_testdir='bootstrap'
| abs_builddir='/tmp/guix-build-kyua-0.13.drv-0/source/bootstrap'
| at_srcdir='.'
| abs_srcdir='/tmp/guix-build-kyua-0.13.drv-0/source/bootstrap'
| at_top_srcdir='..'
| abs_top_srcdir='/tmp/guix-build-kyua-0.13.drv-0/source'
| at_top_build_prefix='../'
| abs_top_builddir='/tmp/guix-build-kyua-0.13.drv-0/source'
| 
| # Backward compatibility with Autotest <= 2.59b:
| at_top_builddir=$at_top_build_prefix
| 
| AUTOTEST_PATH='bootstrap'
| 
| SHELL=${CONFIG_SHELL-'/gnu/store/p7zqfnpxkqj7im10x2ncpm9l62g3zmnp-bash-minimal-5.1.16/bin/bash'}

## ---------------- ##
## Tested programs. ##
## ---------------- ##

./testsuite.at:29: /tmp/guix-build-kyua-0.13.drv-0/source/kyua --version
Usage error: Unknown option --version.
Type 'kyua help' for usage information.

## ------------------ ##
## Running the tests. ##
## ------------------ ##
testsuite: starting at: Mon Jan 29 15:46:18 UTC 2024
1. test program crashes in test list (testsuite.at:72): ok     (0m0.021s 0m0.010s)
2. test program prints an empty test list (testsuite.at:83): ok     (0m0.023s 0m0.010s)
3. test program with zero test cases (testsuite.at:94): ok     (0m0.018s 0m0.016s)
4. run test case that passes (testsuite.at:105): ok     (0m0.024s 0m0.016s)
5. run test case that fails (testsuite.at:117): ok     (0m0.032s 0m0.007s)
6. run test case that skips (testsuite.at:130): ok     (0m0.019s 0m0.021s)
7. run two test cases, success (testsuite.at:143): ok     (0m0.029s 0m0.017s)
8. run two test cases, failure (testsuite.at:156): ok     (0m0.021s 0m0.023s)
9. run mixed test cases (testsuite.at:169): ok     (0m0.031s 0m0.020s)
10. run tests from build directories (testsuite.at:183): ok     (0m0.042s 0m0.071s)
testsuite: ending at: Mon Jan 29 15:46:19 UTC 2024
testsuite: test suite duration: 0h 0m 1s

## ------------- ##
## Test results. ##
## ------------- ##

All 10 tests were successful.

error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("check" "-j" "4") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `check' failed after 58.9 seconds

And to more easily analyze in isolation what is going on, I'd like to run a single test, for example the verbose test case of the cmd_report_test suite. Is this possible?

Thank you!

Apteryks commented 7 months ago

After more investigation, these 4 failures are a side effect of a problem building ATF using the ATF_SHELL configure flag. Will open an issue with ATF.