installed tests fail with pseudoterminal active

[jtojnar]

I am trying to make installed tests pass on NixOS but cannot move any further with the following five:

flatpak/test-pull-bareuseronly.sh.test (Child process exited with code 1)
FAIL: flatpak/test-pull-bareuser.sh.test (Child process exited with code 1)
FAIL: flatpak/test-pull-bare.sh.test (Child process exited with code 1)
FAIL: flatpak/test-remote-add.sh.test (Child process exited with code 1)
FAIL: flatpak/test-gpg-verify-result.test

See the detailed logs for more information.

Build details

    libostree 2018.5 (release)
    features:  libsoup gpgme libarchive selinux libmount release
    ===============

    introspection:                                yes
    Rust (internal oxidation):                    no
    rofiles-fuse:                                 yes
    HTTP backend:                                 libsoup
    "ostree trivial-httpd":                       yes
    SELinux:                                      yes
    cryptographic checksums:                      glib
    systemd:                                      yes
    libmount:                                     yes
    libarchive (parse tar files directly):        yes
    static deltas:                                yes (always enabled now)
    O_TMPFILE:                                    yes
    wrpseudo-compat:                              no
    man pages (xsltproc):                         yes
    api docs (gtk-doc):                           no
    installed tests:                              yes
    gjs-based tests:                              yes
    dracut:                                       no
    mkinitcpio:                                   no
    Static compiler for ostree-prepare-root:      no
    Experimental API                              no
    grub2-mkconfig path:                          grub2-mkconfig

[jlebon]

Hmm odd, it looks like bogus regex matching failures:

++ ls -al err.txt
-rw-r--r-- 1 root root 54 May 18 23:54 err.txt
++ sed -e 's/^/# /'
# error: Invalid ref name origin:main
++ fatal 'File '\''err.txt'\'' doesn'\''t match regexp '\''error: Invalid ref name origin:main'\'''
++ echo File ''\''err.txt'\''' 'doesn'\''t' match regexp ''\''error:' Invalid ref name 'origin:main'\'''
File 'err.txt' doesn't match regexp 'error: Invalid ref name origin:main'
++ exit 1
Running test: flatpak/test-remote-add.sh.test

I wonder if it's related to:

/nix/store/bm8fxl3vjfx1062mw9y65j3ym37sgap2-ostree-2018.5-installedTests/libexec/installed-tests/flatpak/tests/libtest-core.sh: line 43: warning: setlocale: LC_ALL: cannot change locale (C.utf8): No such file or directory

Maybe try setting TEST_SKIP_CLEANUP=1 and inspecting the txt file and playing around with grep?

[jtojnar]

One issue with the tests is that they use C.utf8 locale even when it does not exist

https://github.com/ostreedev/ostree/blob/bf3525adcbdcafa67597c216ed4465d4bceadd54/tests/libtest-core.sh#L42

$ locale -a | grep C.utf8
en_SC.utf8
es_EC.utf8

[jtojnar]

I fixed the locale issue in #1611 but the tests are still failing.

[jtojnar]

Apparently, the issue is caused by escape codes:

++ assert_file_has_content err.txt 'error: Invalid ref name origin:main'
++ fpath=err.txt
++ shift
++ for re in "$@"
++ grep -q -e 'error: Invalid ref name origin:main' err.txt
++ hexdump --canonical err.txt
00000000  1b 5b 33 31 6d 1b 5b 31  6d 65 72 72 6f 72 3a 1b  |.[31m.[1merror:.|
00000010  5b 30 6d 1b 5b 32 32 6d  20 49 6e 76 61 6c 69 64  |[0m.[22m Invalid|
00000020  20 72 65 66 20 6e 61 6d  65 20 6f 72 69 67 69 6e  | ref name origin|
00000030  3a 6d 61 69 6e 0a                                 |:main.|
00000036

(full log)

[cgwalters]

Hmm...is your build process providing a pseudoterminal? OK I see, it's because we're only detecting if stdout is a tty, which in your case it will be.

Probably the most robust fix is to add >/dev/null to all of the calls that are doing 2>err.txt, although I wonder if at the same time we should add a helper function for this, something like:

assert_fails_with 'error: Invalid ref name origin:main' ostree --repo=mirrorrepo-local pull-local otherrepo or so.

[jtojnar]

Apparently, test-gpg-verify-result.test is a separate issue, opened https://github.com/ostreedev/ostree/issues/1634

[jtojnar]

@cgwalters The installed tests are run using gnome-desktop-testing-runner not sure if it runs under pseudo-terminal.

[jtojnar]

And apparently, test-fsck-collections.sh installed test, which passed in 2018.5, fails on current master (cc5254ac34219e187e6963314248308e4a396152):

Build log

``` machine# Running test: libostree/test-fsck-collections.sh.test machine# Copying gpghome to /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# machine# # testlabel.txt: security.selinux: No such attribute machine# Unable to retrieve SELinux label, assuming disabled machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# done machine# ++ BUILT_WITH_ASAN= machine# ++ test -n '' machine# ++ test -z '' machine# ++ test -z '' machine# ++ CMD_PREFIX='env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so' machine# ++ test -n '' machine# ++ '[' -x /nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/../../libostree/ostree-trivial-httpd ']' machine# ++ OSTREE_HTTPD='env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so /nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/../../libostree/ostree-trivial-httpd' machine# ++ _have_user_xattrs= machine# machine# + echo 1..11 machine# forked to background, child pid 758+ cd /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# machine# [ 6.480752] + set_up_repo_with_collection_id machine# systemd-logind[534]: + rm -rf repo files machine# Watching system buttons on /dev/input/event2 (Power Button) machine# + mkdir repo machine# + ostree_repo_init repo --collection-id org.example.Collection machine# + repo=repo machine# + shift machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=repo init --collection-id org.example.Collection machine# + test -n 1 machine# + echo -e 'disable-xattrs=true\n' machine# + mkdir files machine# + pushd files machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ/files /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo commit -s 'Commit 1' -b ref1 machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo commit -s 'Commit 2' --orphan --bind-ref ref2 --add-metadata-string=ostree.collection-binding=org.example.Collection2 machine# ++ cat ../ref2-checksum machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo refs --collections --create=org.example.Collection2:ref2 d21645cfecc19a22f4f84b75ac5f4799376e7decdb19b78ca123045546e96e32 machine# + popd machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree fsck --repo=repo machine# + assert_file_has_content fsck '^Validating refs in collections...$' machine# + fpath=fsck machine# + shift machine# + for re in "$@" machine# + grep -q -e '^Validating refs in collections...$' fsck machine# + wc -l machine# + find repo/objects -name '*.commit' -delete -print machine# + assert_file_has_content commitcount '^2$' machine# + fpath=commitcount machine# + shift machine# + for re in "$@" machine# + grep -q -e '^2$' commitcount machine# + rm repo/refs/heads/ref1 machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree fsck --repo=repo machine# Object missing: d21645cfecc19a22f4f84b75ac5f4799376e7decdb19b78ca123045546e96e32.commit machine# error: Loading commit for ref (org.example.Collection2, ref2): No such metadata object d21645cfecc19a22f4f84b75ac5f4799376e7decdb19b78ca123045546e96e32.commit machine# + assert_file_has_content fsck '^Validating refs...$' machine# + fpath=fsck machine# + shift machine# + for re in "$@" machine# + grep -q -e '^Validating refs...$' fsck machine# + assert_file_has_content fsck '^Validating refs in collections...$' machine# + fpath=fsck machine# + shift machine# + for re in "$@" machine# + grep -q -e '^Validating refs in collections...$' fsck machine# + echo 'ok 1 fsck-collections' machine# ok 1 fsck-collections machine# + set_up_repo_with_collection_id machine# + rm -rf repo files machine# + mkdir repo machine# + ostree_repo_init repo --collection-id org.example.Collection machine# + repo=repo machine# + shift machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=repo init --collection-id org.example.Collection machine# + test -n 1 machine# + echo -e 'disable-xattrs=true\n' machine# + mkdir files machine# + pushd files machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ/files /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo commit -s 'Commit 1' -b ref1 machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo commit -s 'Commit 2' --orphan --bind-ref ref2 --add-metadata-string=ostree.collection-binding=org.example.Collection2 machine# ++ cat ../ref2-checksum machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo refs --collections --create=org.example.Collection2:ref2 d21645cfecc19a22f4f84b75ac5f4799376e7decdb19b78ca123045546e96e32 machine# + popd machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# + rm -rf repo/refs/mirrors machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree fsck --repo=repo machine# + assert_file_has_content fsck '^Validating refs...$' machine# + fpath=fsck machine# + shift machine# + for re in "$@" machine# + grep -q -e '^Validating refs...$' fsck machine# + assert_file_has_content fsck '^Validating refs in collections...$' machine# + fpath=fsck machine# + shift machine# + for re in "$@" machine# + grep -q -e '^Validating refs in collections...$' fsck machine# + echo 'ok 2 fsck-collections in old repository' machine# ok 2 fsck-collections in old repository machine# + set_up_repo_with_collection_id machine# + rm -rf repo files machine# + mkdir repo machine# + ostree_repo_init repo --collection-id org.example.Collection machine# + repo=repo machine# + shift machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=repo init --collection-id org.example.Collection machine# + test -n 1 machine# + echo -e 'disable-xattrs=true\n' machine# + mkdir files machine# + pushd files machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ/files /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo commit -s 'Commit 1' -b ref1 machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo commit -s 'Commit 2' --orphan --bind-ref ref2 --add-metadata-string=ostree.collection-binding=org.example.Collection2 machine# ++ cat ../ref2-checksum machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=../repo refs --collections --create=org.example.Collection2:ref2 5dbb65efff91e65e3b31a2d16c09de2235a094e1ea1f156fd89ba32a7486fe23 machine# + popd machine# /tmp/test-tmp-libostree_test-fsck-collections.sh.test-FKTOKZ machine# ++ cat ref1-checksum machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=repo refs --create=new-ref 38e716ecbf00463b2661cee07a27395ea35a53e889d11b325aae2817b75d3db3 machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree fsck --repo=repo machine# Validating refs... machine# Validating refs in collections... machine# Enumerating objects... machine# Verifying content integrity of 2 commit objects... machine# 78fsck objects (1/4) [=== ] 25%8fsck objects (4/4) [=============] 100% machine# + env LD_PRELOAD=/nix/store/bn69ybvdmkxi61al93y2lww9pfr8h11l-ostree-2018.5-installedTests/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree fsck --repo=repo --verify-bindings machine# + assert_file_has_content fsck-error 'Commit has no requested ref ‘new-ref’ in ref binding metadata (‘ref1’)' machine# + fpath=fsck-error machine# + shift machine# + for re in "$@" machine# + grep -q -e 'Commit has no requested ref ‘new-ref’ in ref binding metadata (‘ref1’)' fsck-error machine# + _fatal_print_file fsck-error 'File '\''fsck-error'\'' doesn'\''t match regexp '\''Commit has no requested ref ‘new-ref’ in ref binding metadata (‘ref1’)'\''' machine# + file=fsck-error machine# + shift machine# + ls -al fsck-error machine# -rw-r--r-- 1 root root 151 Jun 19 15:33 fsck-error machine# + sed -e 's/^/# /' machine# # error: Commit 38e716ecbf00463b2661cee07a27395ea35a53e889d11b325aae2817b75d3db3: Commit has no requested ref ?new-ref? in ref binding metadata (?ref1?) machine# + fatal 'File '\''fsck-error'\'' doesn'\''t match regexp '\''Commit has no requested ref ‘new-ref’ in ref binding metadata (‘ref1’)'\''' machine# + echo File ''\''fsck-error'\''' 'doesn'\''t' match regexp ''\''Commit' has no requested ref $'\342\200\230new-ref\342\200\231' in ref binding metadata '(‘ref1’)'\''' machine# File 'fsck-error' doesn't match regexp 'Commit has no requested ref ‘new-ref’ in ref binding metadata (‘ref1’)' machine# + exit 1 machine# [ 7.880273] gnome-desktop-testing-runner[683]: FAIL: libostree/test-fsck-collections.sh.test (Child process exited with code 1) machine# [ 7.889955] gnome-desktop-testing-runner[683]: SUMMARY: total=1; passed=0; skipped=0; failed=1; user=0.2s; system=0.3s; maxrss=9832 machine# FAIL: libostree/test-fsck-collections.sh.test (Child process exited with code 1) ```

Bisecting it points to 8fbf19c9f54ba34ae7ebbc540b08566b8dd18848

But it looks like the same issue as this one, only with a different file name.

Bisection set-up

#### `/media/Projects/ostree/bisect.sh`: ```shell #!/usr/bin/env bash nix build -f '' tests.ostree.x86_64-linux ``` #### `/media/Projects/nixpkgs/nixos/tests/ostree.nix` ```nix # run installed tests import ./make-test.nix ({ pkgs, lib, ... }: { name = "ostree"; meta = { maintainers = pkgs.ostree.meta.maintainers; }; # TODO: Wrap/patch the tests directly in the package machine = { pkgs, ... }: { environment.systemPackages = with pkgs; [ gnome-desktop-testing ostree gnupg (python3.withPackages (p: with p; [ pyyaml ])) ]; environment.variables.GI_TYPELIB_PATH = lib.makeSearchPath "lib/girepository-1.0" (with pkgs; [ gtk3 pango.out ostree gdk_pixbuf atk ]); # for GJS tests }; testScript = '' $machine->succeed("OT_TESTS_DEBUG=1 gnome-desktop-testing-runner -p 1 -d ${pkgs.ostree.installedTests}/share libostree/test-fsck-collections.sh.test 1>&2"); ''; }) ``` #### `/media/Projects/nixpkgs/pkgs/tools/misc/ostree/default.nix` ```nix { stdenv, fetchFromGitHub, fetchpatch, pkgconfig, gtk-doc, gobjectIntrospection, gnome3 , glib, systemd, xz, e2fsprogs, libsoup, gpgme, which, autoconf, automake, libtool, fuse, utillinuxMinimal, libselinux , libarchive, libcap, bzip2, yacc, libxslt, docbook_xsl, docbook_xml_dtd_42, python3 }: let version = "2018.5"; libglnx-src = fetchFromGitHub { owner = "GNOME"; repo = "libglnx"; rev = "e1a78cf2f5351d5394ccfb79f3f5a7b4917f73f3"; sha256 = "10kzyjbrmr98i65hlz8jc1v5bijyqwwfp6qqjbd5g3y0n520iaxc"; }; bsdiff-src = fetchFromGitHub { owner = "mendsley"; repo = "bsdiff"; rev = "1edf9f656850c0c64dae260960fabd8249ea9c60"; sha256 = "1h71d2h2d3anp4msvpaff445rnzdxii3id2yglqk7af9i43kdsn1"; }; in stdenv.mkDerivation { name = "ostree-${version}"; outputs = [ "out" "dev" "man" "installedTests" ]; src = builtins.fetchGit "/media/Projects/ostree"; nativeBuildInputs = [ autoconf automake libtool pkgconfig gtk-doc gobjectIntrospection which yacc libxslt docbook_xsl docbook_xml_dtd_42 ]; buildInputs = [ glib systemd e2fsprogs libsoup gpgme fuse libselinux libcap libarchive bzip2 xz utillinuxMinimal # for libmount (python3.withPackages (p: with p; [ pyyaml ])) gnome3.gjs # for tests ]; prePatch = '' rmdir libglnx bsdiff cp --no-preserve=mode -r ${libglnx-src} libglnx cp --no-preserve=mode -r ${bsdiff-src} bsdiff ''; preConfigure = '' env NOCONFIGURE=1 ./autogen.sh ''; configureFlags = [ "--with-systemdsystemunitdir=$(out)/lib/systemd/system" "--with-systemdsystemgeneratordir=$(out)/lib/systemd/system-generators" "--enable-installed-tests" ]; makeFlags = [ "installed_testdir=$(installedTests)/libexec/installed-tests/libostree" "installed_test_metadir=$(installedTests)/share/installed-tests/libostree" ]; postInstall = '' moveToOutput libexec/libostree/ostree-trivial-httpd $installedTests ''; meta = with stdenv.lib; { description = "Git for operating system binaries"; homepage = https://ostree.readthedocs.io/en/latest/; license = licenses.lgpl2Plus; platforms = platforms.linux; maintainers = with maintainers; [ copumpkin ]; }; } ```