atomic install crash

[olivier-dj]

Hello, Here is my set of commands on a fresh Fedora 28 VM

curl -LO https://kojipkgs.fedoraproject.org//packages/kernel/4.16.11/300.fc28/x86_64/kernel-headers-4.16.11-300.fc28.x86_64.rpm curl -LO https://kojipkgs.fedoraproject.org//packages/kernel/4.16.11/300.fc28/x86_64/kernel-devel-4.16.11-300.fc28.x86_64.rpm

rpm-ostree install kernel*

echo "blacklist nouveau" >> /etc/modprobe.d/blacklist-nouveau.conf && \ echo "options nouveau modeset=0" >> /etc/modprobe.d/blacklist-nouveau.conf && \ rpm-ostree initramfs --arg=rd.driver.blacklist=nouveau --enable && reboot

rpm-ostree install pciutils wget dkms perl-Getopt-Long elfutils-libelf-devel mesa-libGLU-devel freeglut-devel libXi-devel libXmu-devel make

setenforce 0 && \ sed -i -e 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config && \ systemctl stop docker && systemctl disable docker && \ atomic install --system --system-package no --storage ostree --name docker mycontainerpath && \ service docker restart

And Here is the Error I get, I feel like it not 100% deterministic in the sense that I did manage to install the container without this error, jsut doing the previous steps in a different order, but didn't manage to reproduce. If I retry the same commands just after getting the error it works.

=====

Storing signatures

(process:1310): OSTree-CRITICAL **: 13:12:45.242: pop_repo_lock: assertion 'lock_table != NULL' failed

(process:1310): GLib-CRITICAL : 13:12:45.243: g_propagate_error: assertion 'src != NULL' failed ERROR:/builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/ostreedev/ostree-go/pkg/glibobject/glibobject.go.h:6:_g_error_get_message: assertion failed: (error != NULL) SIGABRT: abort PC=0x7fd965bf3f2b m=5 sigcode=18446744073709551610

goroutine 0 [idle]: runtime: unknown pc 0x7fd965bf3f2b stack: frame={sp:0x7fd95cd18940, fp:0x0} stack=[0x7fd95c519090,0x7fd95cd18c90) 00007fd95cd18840: 00007fd94858b930 00007fd94865d000 00007fd95cd18850: 00000000000000d6 00007fd94858b930 00007fd95cd18860: 00007fd94865d000 0000000000000001 00007fd95cd18870: 00007fd95cd188e0 00007fd96663336a 00007fd95cd18880: 0000000000092d50 00007fd95cd188c8 00007fd95cd18890: 00007fd95cd188d0 00007fd95cd189f8 00007fd95cd188a0: 00007fd9484e26d0 204c273bde2aec00 00007fd95cd188b0: 0000000000000000 204c273bde2aec00 00007fd95cd188c0: 00007fd94858b930 00007fd9483ad690 00007fd95cd188d0: 00007fd9483ad690 0000000000000002 00007fd95cd188e0: 0000000000000000 0000000000000000 00007fd95cd188f0: 00000000000000d2 00007fd965be245f 00007fd95cd18900: 00007fd94858b930 00007fd94865d000 00007fd95cd18910: 00007fd95cd189f8 00007fd965be09cf 00007fd95cd18920: 00000000000000d2 00007fd9666335e7 00007fd95cd18930: 00007fd9484e2735 00000000000000d2 00007fd95cd18940: <0000000000000000 000056516ca08070 00007fd95cd18950: 00007fd9484e26d0 00007fd96669e8ec 00007fd95cd18960: 0000000000000000 00007fd9666338cd 00007fd95cd18970: 00007fd95cd189f8 00007fd965ca80dc 00007fd95cd18980: 0000000000000000 00000000000000d2 00007fd95cd18990: 00000000000000d2 00007fd94865d000 00007fd95cd189a0: 0000000000000800 0000000000000000 00007fd95cd189b0: 0000000000000007 0000000000000000 00007fd95cd189c0: fffffffe7fffffff ffffffffffffffff 00007fd95cd189d0: ffffffffffffffff ffffffffffffffff 00007fd95cd189e0: ffffffffffffffff ffffffffffffffff 00007fd95cd189f0: ffffffffffffffff ffffffffffffffff 00007fd95cd18a00: ffffffffffffffff ffffffffffffffff 00007fd95cd18a10: ffffffffffffffff ffffffffffffffff 00007fd95cd18a20: ffffffffffffffff ffffffffffffffff 00007fd95cd18a30: ffffffffffffffff ffffffffffffffff runtime: unknown pc 0x7fd965bf3f2b stack: frame={sp:0x7fd95cd18940, fp:0x0} stack=[0x7fd95c519090,0x7fd95cd18c90) 00007fd95cd18840: 00007fd94858b930 00007fd94865d000 00007fd95cd18850: 00000000000000d6 00007fd94858b930 00007fd95cd18860: 00007fd94865d000 0000000000000001 00007fd95cd18870: 00007fd95cd188e0 00007fd96663336a 00007fd95cd18880: 0000000000092d50 00007fd95cd188c8 00007fd95cd18890: 00007fd95cd188d0 00007fd95cd189f8 00007fd95cd188a0: 00007fd9484e26d0 204c273bde2aec00 00007fd95cd188b0: 0000000000000000 204c273bde2aec00 00007fd95cd188c0: 00007fd94858b930 00007fd9483ad690 00007fd95cd188d0: 00007fd9483ad690 0000000000000002 00007fd95cd188e0: 0000000000000000 0000000000000000 00007fd95cd188f0: 00000000000000d2 00007fd965be245f 00007fd95cd18900: 00007fd94858b930 00007fd94865d000 00007fd95cd18910: 00007fd95cd189f8 00007fd965be09cf 00007fd95cd18920: 00000000000000d2 00007fd9666335e7 00007fd95cd18930: 00007fd9484e2735 00000000000000d2 00007fd95cd18940: <0000000000000000 000056516ca08070 00007fd95cd18950: 00007fd9484e26d0 00007fd96669e8ec 00007fd95cd18960: 0000000000000000 00007fd9666338cd 00007fd95cd18970: 00007fd95cd189f8 00007fd965ca80dc 00007fd95cd18980: 0000000000000000 00000000000000d2 00007fd95cd18990: 00000000000000d2 00007fd94865d000 00007fd95cd189a0: 0000000000000800 0000000000000000 00007fd95cd189b0: 0000000000000007 0000000000000000 00007fd95cd189c0: fffffffe7fffffff ffffffffffffffff 00007fd95cd189d0: ffffffffffffffff ffffffffffffffff 00007fd95cd189e0: ffffffffffffffff ffffffffffffffff 00007fd95cd189f0: ffffffffffffffff ffffffffffffffff 00007fd95cd18a00: ffffffffffffffff ffffffffffffffff 00007fd95cd18a10: ffffffffffffffff ffffffffffffffff 00007fd95cd18a20: ffffffffffffffff ffffffffffffffff 00007fd95cd18a30: ffffffffffffffff ffffffffffffffff

goroutine 1 [syscall]: runtime.cgocall(0x56516c96dd00, 0xc4200ef070, 0xffffffffffffffff) /usr/lib/golang/src/runtime/cgocall.go:128 +0x66 fp=0xc4200ef040 sp=0xc4200ef008 pc=0x56516c36ae96 github.com/ostreedev/ostree-go/pkg/glibobject._Cfunc__g_error_get_message(0x0, 0x0) _cgo_gotypes.go:151 +0x4c fp=0xc4200ef070 sp=0xc4200ef040 pc=0x56516c8494cc github.com/ostreedev/ostree-go/pkg/glibobject.ConvertGError.func2(0x0, 0x56516ce358e0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/ostreedev/ostree-go/pkg/glibobject/gerror.go:70 +0x58 fp=0xc4200ef0a8 sp=0xc4200ef070 pc=0x56516c8497f8 github.com/ostreedev/ostree-go/pkg/glibobject.ConvertGError(0x0, 0x0, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/ostreedev/ostree-go/pkg/glibobject/gerror.go:70 +0x7a fp=0xc4200ef0e0 sp=0xc4200ef0a8 pc=0x56516c84966a github.com/ostreedev/ostree-go/pkg/otbuiltin.generateError(0x0, 0xc42030e100, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/ostreedev/ostree-go/pkg/otbuiltin/builtin.go:86 +0x42 fp=0xc4200ef198 sp=0xc4200ef0e0 pc=0x56516c84c4e2 github.com/ostreedev/ostree-go/pkg/otbuiltin.(Repo).CommitTransaction(0xc42000e078, 0xc42000e078, 0xc42033a180, 0x3d) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/ostreedev/ostree-go/pkg/otbuiltin/commit.go:86 +0xb0 fp=0xc4200ef1e0 sp=0xc4200ef198 pc=0x56516c84c860 github.com/containers/image/ostree.(ostreeImageDestination).Commit(0xc42008c8f0, 0x0, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/containers/image/ostree/ostree_dest.go:457 +0x769 fp=0xc4200ef398 sp=0xc4200ef1e0 pc=0x56516c8648b9 github.com/containers/image/copy.Image(0xc4204c4380, 0x56516ce45200, 0xc4204b84b0, 0x56516ce44fc0, 0xc4204c80f0, 0xc4206a9670, 0x0, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/containers/image/copy/copy.go:179 +0x4cd fp=0xc4200ef560 sp=0xc4200ef398 pc=0x56516c627f9d main.copyHandler(0xc42049ea00, 0x0, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/cmd/skopeo/copy.go:74 +0x620 fp=0xc4200ef6d8 sp=0xc4200ef560 pc=0x56516c965210 runtime.call32(0xc4204b81b0, 0x56516ce35e48, 0xc4204c40a0, 0x800000018) /usr/lib/golang/src/runtime/asm_amd64.s:573 +0x3d fp=0xc4200ef708 sp=0xc4200ef6d8 pc=0x56516c3be54d reflect.Value.call(0x56516cd34440, 0x56516ce35e48, 0x13, 0x56516c96ef9b, 0x4, 0xc420169ab0, 0x1, 0x1, 0xc4200661c0, 0x56516ce1c060, ...) /usr/lib/golang/src/reflect/value.go:447 +0x96b fp=0xc4200ef9c8 sp=0xc4200ef708 pc=0x56516c41c94b reflect.Value.Call(0x56516cd34440, 0x56516ce35e48, 0x13, 0xc420169ab0, 0x1, 0x1, 0x4, 0xc420453118, 0x5) /usr/lib/golang/src/reflect/value.go:308 +0xa6 fp=0xc4200efa30 sp=0xc4200ef9c8 pc=0x56516c41bea6 github.com/urfave/cli.HandleAction(0x56516cd34440, 0x56516ce35e48, 0xc42049ea00, 0x0, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/urfave/cli/app.go:480 +0x175 fp=0xc4200efaf8 sp=0xc4200efa30 pc=0x56516c956f45 github.com/urfave/cli.Command.Run(0x56516c96f0b3, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x56516c98d5eb, 0x2f, 0x0, ...) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/urfave/cli/command.go:186 +0x99b fp=0xc4200efd88 sp=0xc4200efaf8 pc=0x56516c9582fb github.com/urfave/cli.(*App).Run(0xc420001e00, 0xc42001e0c0, 0x6, 0x6, 0x0, 0x0) /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/vendor/src/github.com/urfave/cli/app.go:236 +0x548 fp=0xc4200eff38 sp=0xc4200efd88 pc=0x56516c9550c8 main.main() /builddir/build/BUILD/skopeo-7add6fc80b0f33406217e7c3361cb711c814f028/cmd/skopeo/main.go:91 +0x5e fp=0xc4200eff88 sp=0xc4200eff38 pc=0x56516c96617e runtime.main() /usr/lib/golang/src/runtime/proc.go:198 +0x21a fp=0xc4200effe0 sp=0xc4200eff88 pc=0x56516c3956da runtime.goexit() /usr/lib/golang/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4200effe8 sp=0xc4200effe0 pc=0x56516c3c0d51

goroutine 5 [syscall, 1 minutes]: os/signal.signal_recv(0x0) /usr/lib/golang/src/runtime/sigqueue.go:139 +0xa8 os/signal.loop() /usr/lib/golang/src/os/signal/signal_unix.go:22 +0x24 created by os/signal.init.0 /usr/lib/golang/src/os/signal/signal_unix.go:28 +0x43

rax 0x0 rbx 0x6 rcx 0x7fd965bf3f2b rdx 0x0 rdi 0x2 rsi 0x7fd95cd18940 rbp 0xcf rsp 0x7fd95cd18940 r8 0x0 r9 0x7fd95cd18940 r10 0x8 r11 0x246 r12 0x7fd96691ce20 r13 0x7fd95cd18b90 r14 0x56516ca08100 r15 0x200 rip 0x7fd965bf3f2b rflags 0x246 cs 0x33 fs 0x0 gs 0x0

[razorfish-sl]

yep I'm seeing the same on openstack queens & cloud-init. but it is repeatable.

I have a full dmesg, cloud-init,journal.

Jun 24 08:26:33.861136 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: Storing signatures Jun 24 08:28:11.976528 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: (process:1363): OSTree-CRITICAL : 08:28:11.974: pop_repo_lock: assertion 'lock_table != NULL' failed Jun 24 08:28:11.979819 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: (process:1363): GLib-CRITICAL : 08:28:11.979: g_propagate_error: assertion 'src != NULL' failed Jun 24 08:28:11.981075 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: ** Jun 24 08:28:11.981075 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: ERROR:/builddir/build/BUILD/skopeo-5c611083f263f330f396772ed9dd7d25299e20a0/vendor/src/github.com/ostreedev/ostree-go/pkg/glibobject/glibobject.go.h:6:_g_error_get_message: assertion failed: (error != NULL) Jun 24 08:28:11.983815 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: SIGABRT: abort Jun 24 08:28:11.983815 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: PC=0x7fe52a8a1f2b m=6 sigcode=18446744073709551610 Jun 24 08:28:11.983815 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: goroutine 0 [idle]: Jun 24 08:28:11.983815 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: runtime: unknown pc 0x7fe52a8a1f2b Jun 24 08:28:11.986063 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: stack: frame={sp:0x7fe5211c3940, fp:0x0} stack=[0x7fe5209c4090,0x7fe5211c3c90) Jun 24 08:28:11.986063 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: 00007fe5211c3840: 00007fe510231820 00007fe510256e90 Jun 24 08:28:11.986063 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: 00007fe5211c3850: 00000000000000d6 00007fe510231820 Jun 24 08:28:11.986063 sdsds-5jnousguo6qs-master-0.novalocal cloud-init[1131]: 00007fe5211c3860: 00007fe510256e90 0000000000000001 Jun 24 08:28:11.9 .... etc

[giuseppe]

the issue is in containers/image (used by Skopeo to pull images from a registry) and it was caused by a recent change in ostree: https://github.com/containers/image/pull/461.

This was already fixed on Fedora with a new version of Skopeo. Are you using the latest version of Skopeo available on F28? https://bodhi.fedoraproject.org/updates/FEDORA-2018-ce7128a8d1 ?

[Hardcore-fs]

I'm Just rebuilding my openstack incase it was that...... now I know it is not..... I will pull this new F28 image and give it a spin but it will take a bit of time as i'm not really into Fedora. i was using https://dl.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-28-20180613.1/

since that is the latest "atomic" available.