search

ostreedev / ostree

Operating system and container binary deployment and upgrades
https://ostreedev.github.io/ostree/
Other
1.26k stars 291 forks source link

Tests fail in some environments: assertion failed: (changed) #1885

Open smcv opened 5 years ago

smcv commented 5 years ago

Debian's new pre-upload CI infrastructure runs tests in at least these two situations:

Some tests fail with errors like this one (this is from the installed-tests, but the build-time tests were similar):

PASS: libostree/test-repo.test
Running test: libostree/test-sysroot-c.test
+ . /usr/libexec/installed-tests/libostree/tests/libtest.sh
+++ dirname bash
++ dn=.
++ '[' -n /usr/libexec/installed-tests/libostree ']'
++ test_srcdir=/usr/libexec/installed-tests/libostree/tests
++ '[' -n /usr/libexec/installed-tests/libostree ']'
++ test_builddir=/usr/libexec/installed-tests/libostree/tests
++ . /usr/libexec/installed-tests/libostree/tests/libtest-core.sh
+++ locale -a
+++ grep '^C.UTF-8$'
+++ export LC_ALL=C.UTF-8
+++ LC_ALL=C.UTF-8
+++ unset LANGUAGE
+++ export G_DEBUG=fatal-warnings
+++ G_DEBUG=fatal-warnings
++ trap save_core EXIT
+++ pwd
++ test_tmpdir=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
++ test -f .testtmp
++ unset TAR_OPTIONS
++ export OSTREE_SYSROOT_DEBUG=mutable-deployments
++ OSTREE_SYSROOT_DEBUG=mutable-deployments
++ export TEST_GPG_KEYID_1=472CDAFA
++ TEST_GPG_KEYID_1=472CDAFA
++ export TEST_GPG_KEYID_2=CA950D41
++ TEST_GPG_KEYID_2=CA950D41
++ export TEST_GPG_KEYID_3=DF444D67
++ TEST_GPG_KEYID_3=DF444D67
++ echo 'Copying gpghome to /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z'
Copying gpghome to /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
++ cp -a /usr/libexec/installed-tests/libostree/tests/gpghome /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
++ chmod -R u+w /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
++ export TEST_GPG_KEYHOME=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/gpghome
++ TEST_GPG_KEYHOME=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/gpghome
++ export OSTREE_GPG_HOME=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/gpghome/trusted
++ OSTREE_GPG_HOME=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/gpghome/trusted
++ _have_selinux_relabel=
++ echo -n checking for xattrs...
checking for xattrs...++ have_selinux_relabel
++ assert_has_setfattr
++ which setfattr
/usr/bin/setfattr
++ test '' = ''
++ pushd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
++ echo testlabel
++ selinux_xattr=security.selinux
++ getfattr --encoding=base64 -n security.selinux testlabel.txt
+++ sed -e s,security.selinux=,,
+++ grep -E -e '^security.selinux='
++ label=0sc3lzdGVtX3U6b2JqZWN0X3I6c3ZpcnRfbHhjX2ZpbGVfdDpzMAA=
++ setfattr -n security.selinux -v 0sc3lzdGVtX3U6b2JqZWN0X3I6c3ZpcnRfbHhjX2ZpbGVfdDpzMAA= testlabel.txt
+++ pwd
++ echo 'SELinux enabled in /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z, and have privileges to relabel'
SELinux enabled in /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z, and have privileges to relabel
++ _have_selinux_relabel=yes
++ popd
/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
++ test yes = yes
++ echo done
done
++ test -n ''
++ BUILT_WITH_ASAN=
++ test -n ''
++ test -z ''
++ test -z ''
++ CMD_PREFIX='env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so'
++ test -n ''
++ '[' -x /usr/libexec/installed-tests/libostree/tests/../../libostree/ostree-trivial-httpd ']'
++ OSTREE_HTTPD='env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so /usr/libexec/installed-tests/libostree/tests/../../libostree/ostree-trivial-httpd'
++ _have_user_xattrs=
++ _have_strace_fault_injection=
+ setup_os_repository archive syslinux
+ mode=archive
+ shift
+ bootmode=syslinux
+ shift
+ bootdir=usr/lib/modules/3.6.0
++ pwd
+ oldpwd=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ cd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ mkdir testos-repo
+ test -n archive
+ ostree_repo_init testos-repo --mode=archive
+ repo=testos-repo
+ shift
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=testos-repo init --mode=archive
+ test -n ''
+ cd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ mkdir osdata
+ cd osdata
+ kver=3.6.0
+ mkdir -p usr/bin usr/lib/modules/3.6.0 usr/lib/modules/3.6.0 usr/share usr/etc
+ kernel_path=usr/lib/modules/3.6.0/vmlinuz
+ initramfs_path=usr/lib/modules/3.6.0/initramfs.img
+ [[ usr/lib/modules/3.6.0 != usr/lib/modules/* ]]
+ echo 'a kernel'
+ echo 'an initramfs'
++ cut -f 1 -d ' '
++ sha256sum
++ cat usr/lib/modules/3.6.0/vmlinuz usr/lib/modules/3.6.0/initramfs.img
+ bootcsum=7bd091ad6955c0e7ef41bab65adb016275664b55efc0d10a0c8cb263da3c1e4e
+ export bootcsum
+ [[ usr/lib/modules/3.6.0 != usr/lib/modules/* ]]
+ echo 'an executable'
+ echo 'some shared data'
+ echo 'a library'
+ ln -s usr/bin bin
+ cat
+ echo 'a config file'
+ mkdir -p usr/etc/NetworkManager
+ echo 'a default daemon file'
+ mkdir -p usr/etc/testdirectory
+ echo 'a default daemon file'
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/testos-repo commit --add-metadata-string version=1.0.9 -b testos/buildmaster/x86_64-runtime -s Build
450b1c5ec55df82d4f7441a72ee6ce0476f4d92ec414711a3043cab35e2990cc
+ sleep 2
+ echo 'a new executable'
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/testos-repo commit --add-metadata-string version=1.0.10 -b testos/buildmaster/x86_64-runtime -s Build
8412d21da87047334fc1a73b6d47cb7629f59f56cf33ca37c7f94b641df10528
+ cd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ cp -a osdata osdata-devel
+ cd osdata-devel
+ mkdir -p usr/include
+ echo 'a development header'
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/testos-repo commit --add-metadata-string version=1.0.9 -b testos/buildmaster/x86_64-devel -s Build
bf939373e257725711dbfaabdab1b518cb6a65594d6cb84899352e948b36ca42
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree --repo=/tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/testos-repo fsck -q
fsck objects (1/31) 3%
fsck objects (31/31) 100%
+ cd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ mkdir sysroot
+ export OSTREE_SYSROOT=sysroot
+ OSTREE_SYSROOT=sysroot
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree admin init-fs sysroot
+ test -n ''
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so ostree admin os-init testos
ostree/deploy/testos initialized as OSTree root
+ case $bootmode in
+ setup_os_boot_syslinux
+ mkdir -p sysroot/boot/loader.0
+ ln -s loader.0 sysroot/boot/loader
+ touch sysroot/boot/loader/syslinux.cfg
+ mkdir -p sysroot/boot/syslinux
+ ln -s ../loader/syslinux.cfg sysroot/boot/syslinux/syslinux.cfg
+ cd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ mkdir /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/httpd
+ cd httpd
+ ln -s /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z ostree
+ env LD_PRELOAD=/usr/libexec/installed-tests/libostree/tests/libreaddir-rand.so /usr/libexec/installed-tests/libostree/tests/../../libostree/ostree-trivial-httpd --autoexit --daemonize -p /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/httpd-port
++ cat /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z/httpd-port
+ port=35247
+ echo http://127.0.0.1:35247
+ cd /tmp/test-tmp-libostree_test-sysroot-c.test-F3BR4Z
+ save_core
+ '[' -e core ']'
/sysroot-reload: 12 metadata, 9 content objects imported
Relabeling /var (no stamp file 'var/.ostree-selabeled' found)
Transaction complete; bootconfig swap: yes; deployment count change: 1
**
OSTree:ERROR:tests/test-sysroot-c.c:64:test_sysroot_reload: assertion failed: (changed)
FAIL: libostree/test-sysroot-c.test (Child process killed by signal 6)

I think this might be because overlay mounts break libostree's assumption that altering ostree/deploy/* causes the mtime of ostree/deploy/ to be updated?

smcv commented 5 years ago

Unfortunately I can't reproduce this by running the tests in a Docker container in a local VM, using overlay mounts of an ext4 filesystem.