search

coreos / rpm-ostree

⚛📦 Hybrid image/package system with atomic upgrades and package layering
https://coreos.github.io/rpm-ostree
Other
862 stars 195 forks source link

`rpm-ostree override replace --experimental --freeze --from repo=...` crashes when called with a lot of packages #3344

Closed travier closed 2 years ago

travier commented 2 years ago

Host system details

Latest Fedora Kinoite 35, thus rpm-ostreee from Fedora 35.

$ rpm-ostree --version
rpm-ostree:
 Version: '2022.1'
 Git: 058b71be8eba61ab5c9cebdd6e44da37ed697950
 Features:
  - rust
  - compose
  - fedora-integration

Expected vs actual behavior

I was writing a new blog post and I tried running:

$ sudo rpm-ostree override replace --experimental <(cat packages.list) --freeze --from repo=copr:copr.fedorainfracloud.org:marcdeop:frameworks --from repo=copr:copr.fedorainfracloud.org:marcdeop:plasma
...

With the mentioned copr repos added on the host and a long list of packages in the list.

This crashed the daemon right after fetching the repo metadata.

Coredump info

From coredumpctl:

           PID: 5203 (rpm-ostree)
           UID: 0 (root)
           GID: 0 (root)
        Signal: 6 (ABRT)
     Timestamp: Wed 2022-01-19 14:49:19 CET (6h ago)
  Command Line: /usr/bin/rpm-ostree start-daemon
    Executable: /usr/bin/rpm-ostree
 Control Group: /system.slice/rpm-ostreed.service
          Unit: rpm-ostreed.service
         Slice: system.slice
       Boot ID: 4a8a70507f384b77b82a6919a70a7225
    Machine ID: e8763008a52045dea49f9425598565b5
      Hostname: fedora
       Storage: /var/lib/systemd/coredump/core.rpm-ostree.0.4a8a70507f384b77b82a6919a70a7225.5203.1642600159000000.zst (present)
     Disk Size: 26.0M
       Message: Process 5203 (rpm-ostree) of user 0 dumped core.

                Found module linux-vdso.so.1 with build-id: 32d7f82fe39a1f97afa6f2146152d25fc2874487
                Found module libgdbm.so.6 with build-id: 8099c5fcea87470d071e35f0bb30a107b37f17f1
                Found module libsasldb.so with build-id: 0e1a54aa5856dcc3210cecdb2f0e94cdc1b1ea51
                Found module libplain.so with build-id: cd856b8306754f71764c8e6143c4ec916df6d00c
                Found module liblogin.so with build-id: 7857c20347a3e0af9994e631f1507e919e6a56fc
                Found module libgssapiv2.so with build-id: 604a2a4980744be033a69e2435bd74648824d5c2
                Found module libanonymous.so with build-id: 9c3155a2d81a24f6c078c53e60bf99371ba5f209
                Found module libnss_myhostname.so.2 with build-id: 884539c682a7226ebe838279cb382c786e469e12
                Found module libnss_sss.so.2 with build-id: 63a1597a586dc8670ed177a7f59c4ce5fa54bc34
                Found module libcrypt.so.2 with build-id: 1b61101236df19dcd6b62356a5c786146ac34520
                Found module libassuan.so.0 with build-id: 36e487827c8ec9335bbb100d25524a63cad5e689
                Found module libcap-ng.so.0 with build-id: 1c7599981a7ae118996a9dfaf161e25d430bf14a
                Found module libpcre2-8.so.0 with build-id: 67d78ee981939e51fee33d8885ff8000552e7a3d
                Found module libblkid.so.1 with build-id: 96be27216d8d6d7ba3694ca503cd1b07f60fa539
                Found module libbrotlicommon.so.1 with build-id: 9e92a8ec2d9efe80ec86e04fba0549cd3202ebc4
                Found module libsasl2.so.3 with build-id: 4eefde24d117c2ec7bcd12d4b583ba46571fc22a
                Found module libresolv.so.2 with build-id: 057b93718c09e93c39c7945cdbeb314445b21fd6
                Found module libkeyutils.so.1 with build-id: 2560a16099ad1875f7ea2195ae25b97ea168a758
                Found module libkrb5support.so.0 with build-id: 03649849c4d1813c307624a711b03fb15e7124cb
                Found module libunistring.so.2 with build-id: edcae9db236efd11e61995054ddf27a89ec6dc40
                Found module libattr.so.1 with build-id: 38932d6d81e2a95598ccc83ab14016fa87378891
                Found module libgpg-error.so.0 with build-id: 081975d0a3416374b4883b2f1639fd3c9df42390
                Found module libgcrypt.so.20 with build-id: 711d41580c5a8649a79a8430a985dac3e25b5ba2
                Found module libyaml-0.so.2 with build-id: 48e7fd04f82d348de7b5ac8a23299e73b0f81d41
                Found module libmagic.so.1 with build-id: ec155d5276d51c181fdf4f7def4b68ac0fe77919
                Found module libzck.so.1 with build-id: 4dc530c0689e2d2466664d6111a6175a555190de
                Found module libgpgme.so.11 with build-id: 9448a8d3d1c5efb2da8426c001953ee91b770365
                Found module libaudit.so.1 with build-id: f5aa484ba17b5425d516f2b879d7181e3adb659d
                Found module liblua-5.4.so with build-id: 0ea57f5ce9c67ca9691fa113b478acd6d7ede410
                Found module libpopt.so.0 with build-id: eb8e8dc9ba0c3c2d7c969d4f4805fc410f8d8f22
                Found module libpcre.so.1 with build-id: fcbdb4731e50f20a47b62341a6841dd616fe495d
                Found module libffi.so.6 with build-id: 0129f72b58e11d59546f7f207d2c90af7ebd9a5e
                Found module libselinux.so.1 with build-id: f805394f993c704b949315b56c344d22dfad801f
                Found module libmount.so.1 with build-id: 7f7bccf74cb835b64429e61ea0ec03cd0cb95b8b
                Found module libgmodule-2.0.so.0 with build-id: 634542167d122d8d1aa3b44542dced7bc40d2eb6
                Found module libbrotlidec.so.1 with build-id: 57cb8a53e33f31a620f3739919414c0d00549f87
                Found module liblber-2.4.so.2 with build-id: e7a8b3a354a19afa3f7261c85c4bb30af11f8126
                Found module libldap_r-2.4.so.2 with build-id: 1406008a1002c2265ac3a7e540a5ed4ad278579c
                Found module libcom_err.so.2 with build-id: a1d791cd7600f5609702a895a64d9131d1cd7b8f
                Found module libk5crypto.so.3 with build-id: 11238d51059ecf299699dc4d24a74e67a64aa725
                Found module libkrb5.so.3 with build-id: 0582e999cc7d74e1ec2dc7b3c9f7aa4dae7342f6
                Found module libgssapi_krb5.so.2 with build-id: c79d44cc56f67983ba388e5336c779d9f050114e
                Found module libpsl.so.5 with build-id: 71614cc984977692f16edc6189df04f570f51608
                Found module libssh.so.4 with build-id: 1e466b1f1c44646e8ef4279b82980df32fa03261
                Found module libidn2.so.0 with build-id: f3ad1b0687f5b426beed2d3d3d18aa301932f7a3
                Found module libnghttp2.so.14 with build-id: b6492c1dabf77777b1ae631416297ffc01f7ff30
                Found module libxml2.so.2 with build-id: 8f21e0bbe0106c5c5cd35859487dcc074b8f13bd
                Found module libbz2.so.1 with build-id: 19ce300b78962ee0b113afb0f1016727a2f3c37b
                Found module liblz4.so.1 with build-id: fd02c4542a1ce1ad6c503b958447be1c37a4afee
                Found module libzstd.so.1 with build-id: f25049c14f12e8d3ba5d3e89172180dacc45fbbe
                Found module liblzma.so.5 with build-id: 7fec53ce7cba9489f130c99cfd3ace82e9dde0ee
                Found module libacl.so.1 with build-id: 24b2ced347e35fc1fbc66dfd90c65930ebce8628
                Found module ld-linux-x86-64.so.2 with build-id: b43118df1fdb4c0aff150b6f8f926bccdec2a7f0
                Found module libc.so.6 with build-id: 644dac2c66a6e0b32674f0ec71e7431bd0c06a63
                Found module libgcc_s.so.1 with build-id: 88564abce789aa42536da1247a57ff6062d61dcb
                Found module libostree-1.so.1 with build-id: aa4c20298b815b5fdd9d9ada63ebfa4a345b50a8
                Found module libz.so.1 with build-id: 5903f5c355c264403e4e7cdc66779584425ca3b8
                Found module libsystemd.so.0 with build-id: b61753e8440ecf39624b3e37f855941c18a69e9b
                Found module libsqlite3.so.0 with build-id: 072b1e135c03359061ea9c8346cdb61528950924
                Found module libcrypto.so.1.1 with build-id: 3ead607210606611d5f3398ee7e65198bd519de9
                Found module libssl.so.1.1 with build-id: b3b8eaa0fc316f1bb70ca237310c5b09048d3648
                Found module libmodulemd.so.2 with build-id: 570a8fea5b50f4dc38f84f23f608fb68575e83ae
                Found module libsolv.so.1 with build-id: 05ce8d671486fcb2b2f3d67d86588450b32f84af
                Found module libsolvext.so.1 with build-id: 1a1e9bf4e92f2140c03f4f9325893f699d2d5ec3
                Found module librepo.so.0 with build-id: af06cfd4ad398d62d322beb1fe72ebff42d6f725
                Found module libjson-c.so.5 with build-id: 5ee4e811709779102a0862e28e7f3752c0fdc5d1
                Found module librpmio.so.9 with build-id: 341245651d0f2f1aeddc517b735c4c9ccb4a2827
                Found module librpm.so.9 with build-id: f63625c67e98a2534dfc8bb772515d3f45190ef9
                Found module libglib-2.0.so.0 with build-id: 3720108994016259cac8f61f174ea32513518152
                Found module libgobject-2.0.so.0 with build-id: ead2bc31b68d342e8a9a6a87148862f08dce1bd1
                Found module libgio-2.0.so.0 with build-id: 8be4eeb5ab3e02714b03629052528cdecab1ccfc
                Found module libpolkit-gobject-1.so.0 with build-id: 191b7c001873697be3647ebaa9dddc14744e49ec
                Found module libcurl.so.4 with build-id: dd7faf7805a0041b9267a45b4f039361bb2eb3d5
                Found module libarchive.so.13 with build-id: cf0c29347b92a0b20338dc39d4bec8b19f284397
                Found module libjson-glib-1.0.so.0 with build-id: ff150bb944b7d341fb33c367b230a0c005ac1833
                Found module libm.so.6 with build-id: cea55efc551e62cd8439d044aa9a765c445fb7bf
                Found module libstdc++.so.6 with build-id: 64dc8524cb0109c7f3d3c0ef3d4d6ec9979db75c
                Found module libcap.so.2 with build-id: 0214aa9cc6a8646eb9ec27ab7dda6a2219da6500
                Found module rpm-ostree with build-id: 5c65aa6e2d2e86b6fa74f53cd04ac160474fb2be
                Stack trace of thread 5208:
                #0  0x00007feffacd184c __pthread_kill_implementation (libc.so.6 + 0x8f84c)
                #1  0x00007feffac846a6 raise (libc.so.6 + 0x426a6)
                #2  0x00007feffac6e7d3 abort (libc.so.6 + 0x2c7d3)
                #3  0x00007feffb7a7d4c g_assertion_message.cold (libglib-2.0.so.0 + 0x1ed4c)
                #4  0x00007feffb80845f g_assertion_message_expr (libglib-2.0.so.0 + 0x7f45f)
                #5  0x0000556830775cde _ZL39osexperimental_handle_download_packagesP24_RPMOSTreeOSExperimentalP22_GDBusMethodInvocationP12_GUnixFDListPKPKcS6_ (rpm-ostree + 0x752cde)
                #6  0x00007feffa655c04 ffi_call_unix64 (libffi.so.6 + 0x6c04)
                #7  0x00007feffa655108 ffi_call (libffi.so.6 + 0x6108)
                #8  0x00007feffb8ddbfd g_cclosure_marshal_generic (libgobject-2.0.so.0 + 0x19bfd)
                #9  0x00007feffb8d7c7f g_closure_invoke (libgobject-2.0.so.0 + 0x13c7f)
                #10 0x00007feffb8f3af4 signal_emit_unlocked_R (libgobject-2.0.so.0 + 0x2faf4)
                #11 0x0000556830795fe4 _rpmostree_osexperimental_skeleton_handle_method_call (rpm-ostree + 0x772fe4)
                #12 0x00007feffba43693 g_dbus_interface_method_dispatch_helper (libgio-2.0.so.0 + 0x123693)
                #13 0x00007feffba308ee call_in_idle_cb.lto_priv.0 (libgio-2.0.so.0 + 0x1108ee)
                #14 0x00007feffb7da47b g_idle_dispatch (libglib-2.0.so.0 + 0x5147b)
                #15 0x00007feffb7de130 g_main_context_dispatch (libglib-2.0.so.0 + 0x55130)
                #16 0x00007feffb833208 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xaa208)
                #17 0x00007feffb7db933 g_main_context_iteration (libglib-2.0.so.0 + 0x52933)
                #18 0x000055683075f15c rpmostree_builtin_start_daemon (rpm-ostree + 0x73c15c)
                #19 0x0000556830753b14 _ZN12rpmostreecxx14rpmostree_mainEN4rust10cxxbridge15SliceIKNS1_3StrEEE (rpm-ostree + 0x730b14)
                #20 0x000055683074d10b rpmostreecxx$cxxbridge1$rpmostree_main (rpm-ostree + 0x72a10b)
                #21 0x000055683041ed24 _ZN102_$LT$tokio..runtime..blocking..task..BlockingTask$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$4poll17h6734efc198594498E (rpm-ostree + 0x3fbd24)
                #22 0x000055683040e0f0 _ZN5tokio7runtime4task7harness20Harness$LT$T$C$S$GT$4poll17h6c2d66279090c67eE (rpm-ostree + 0x3eb0f0)
                #23 0x00005568307091c3 _ZN5tokio7runtime8blocking4pool5Inner3run17h991963d9f264a5e7E (rpm-ostree + 0x6e61c3)
                #24 0x00005568306fa74f _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h6f737ab7b0902fdaE (rpm-ostree + 0x6d774f)
                #25 0x000055683070bef4 _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17he4ff7ac48016b426E (rpm-ostree + 0x6e8ef4)
                #26 0x00005568306d6515 _ZN3std3sys4unix6thread6Thread3new12thread_start17hd3aa83834b83bd4aE (rpm-ostree + 0x6b3515)
                #27 0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #28 0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5205:
                #0  0x00007feffad53c1e epoll_wait (libc.so.6 + 0x111c1e)
                #1  0x00005568306fbac6 _ZN5tokio2io6driver6Driver4turn17hbf1535b25f1e4849E.llvm.8263468477234935197 (rpm-ostree + 0x6d8ac6)
                #2  0x0000556830705a3c _ZN5tokio4time6driver15Driver$LT$P$GT$13park_internal17h958c8e8fd8ada213E.llvm.3219000865013277195 (rpm-ostree + 0x6e2a3c)
                #3  0x0000556830701301 _ZN78_$LT$tokio..park..either..Either$LT$A$C$B$GT$$u20$as$u20$tokio..park..Park$GT$4park17ha01702690fa6e83cE (rpm-ostree + 0x6de301)
                #4  0x00005568306feeb4 _ZN66_$LT$tokio..runtime..park..Parker$u20$as$u20$tokio..park..Park$GT$4park17h611b7986a2445208E (rpm-ostree + 0x6dbeb4)
                #5  0x000055683070f119 _ZN5tokio7runtime11thread_pool6worker7Context12park_timeout17hd2aba060b318eeacE (rpm-ostree + 0x6ec119)
                #6  0x000055683070e662 _ZN5tokio7runtime11thread_pool6worker7Context3run17h3d56ab806a126f04E (rpm-ostree + 0x6eb662)
                #7  0x000055683070d746 _ZN5tokio7runtime11thread_pool6worker3run17h909af045f2300022E (rpm-ostree + 0x6ea746)
                #8  0x0000556830706200 _ZN5tokio7runtime4task7harness20Harness$LT$T$C$S$GT$4poll17hf762ac8bbe8eaec8E (rpm-ostree + 0x6e3200)
                #9  0x00005568307091c3 _ZN5tokio7runtime8blocking4pool5Inner3run17h991963d9f264a5e7E (rpm-ostree + 0x6e61c3)
                #10 0x00005568306fa74f _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h6f737ab7b0902fdaE (rpm-ostree + 0x6d774f)
                #11 0x000055683070bef4 _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17he4ff7ac48016b426E (rpm-ostree + 0x6e8ef4)
                #12 0x00005568306d6515 _ZN3std3sys4unix6thread6Thread3new12thread_start17hd3aa83834b83bd4aE (rpm-ostree + 0x6b3515)
                #13 0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #14 0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5209:
                #0  0x00007feffad4773f __poll (libc.so.6 + 0x10573f)
                #1  0x00007feffb83319c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xaa19c)
                #2  0x00007feffb7db933 g_main_context_iteration (libglib-2.0.so.0 + 0x52933)
                #3  0x00007feffb7db981 glib_worker_main (libglib-2.0.so.0 + 0x52981)
                #4  0x00007feffb808842 g_thread_proxy (libglib-2.0.so.0 + 0x7f842)
                #5  0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #6  0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5204:
                #0  0x00007feffad4ce6d syscall (libc.so.6 + 0x10ae6d)
                #1  0x00005568302d6299 _ZN11parking_lot7condvar7Condvar19wait_until_internal17h9b4ec543cacea21cE (rpm-ostree + 0x2b3299)
                #2  0x00005568306fef75 _ZN66_$LT$tokio..runtime..park..Parker$u20$as$u20$tokio..park..Park$GT$4park17h611b7986a2445208E (rpm-ostree + 0x6dbf75)
                #3  0x000055683070f119 _ZN5tokio7runtime11thread_pool6worker7Context12park_timeout17hd2aba060b318eeacE (rpm-ostree + 0x6ec119)
                #4  0x000055683070e662 _ZN5tokio7runtime11thread_pool6worker7Context3run17h3d56ab806a126f04E (rpm-ostree + 0x6eb662)
                #5  0x000055683070d746 _ZN5tokio7runtime11thread_pool6worker3run17h909af045f2300022E (rpm-ostree + 0x6ea746)
                #6  0x0000556830706200 _ZN5tokio7runtime4task7harness20Harness$LT$T$C$S$GT$4poll17hf762ac8bbe8eaec8E (rpm-ostree + 0x6e3200)
                #7  0x00005568307091c3 _ZN5tokio7runtime8blocking4pool5Inner3run17h991963d9f264a5e7E (rpm-ostree + 0x6e61c3)
                #8  0x00005568306fa74f _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h6f737ab7b0902fdaE (rpm-ostree + 0x6d774f)
                #9  0x000055683070bef4 _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17he4ff7ac48016b426E (rpm-ostree + 0x6e8ef4)
                #10 0x00005568306d6515 _ZN3std3sys4unix6thread6Thread3new12thread_start17hd3aa83834b83bd4aE (rpm-ostree + 0x6b3515)
                #11 0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #12 0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5207:
                #0  0x00007feffad4ce6d syscall (libc.so.6 + 0x10ae6d)
                #1  0x00005568302d6299 _ZN11parking_lot7condvar7Condvar19wait_until_internal17h9b4ec543cacea21cE (rpm-ostree + 0x2b3299)
                #2  0x00005568306fef75 _ZN66_$LT$tokio..runtime..park..Parker$u20$as$u20$tokio..park..Park$GT$4park17h611b7986a2445208E (rpm-ostree + 0x6dbf75)
                #3  0x000055683070f119 _ZN5tokio7runtime11thread_pool6worker7Context12park_timeout17hd2aba060b318eeacE (rpm-ostree + 0x6ec119)
                #4  0x000055683070e662 _ZN5tokio7runtime11thread_pool6worker7Context3run17h3d56ab806a126f04E (rpm-ostree + 0x6eb662)
                #5  0x000055683070d746 _ZN5tokio7runtime11thread_pool6worker3run17h909af045f2300022E (rpm-ostree + 0x6ea746)
                #6  0x0000556830706200 _ZN5tokio7runtime4task7harness20Harness$LT$T$C$S$GT$4poll17hf762ac8bbe8eaec8E (rpm-ostree + 0x6e3200)
                #7  0x00005568307091c3 _ZN5tokio7runtime8blocking4pool5Inner3run17h991963d9f264a5e7E (rpm-ostree + 0x6e61c3)
                #8  0x00005568306fa74f _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h6f737ab7b0902fdaE (rpm-ostree + 0x6d774f)
                #9  0x000055683070bef4 _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17he4ff7ac48016b426E (rpm-ostree + 0x6e8ef4)
                #10 0x00005568306d6515 _ZN3std3sys4unix6thread6Thread3new12thread_start17hd3aa83834b83bd4aE (rpm-ostree + 0x6b3515)
                #11 0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #12 0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5203:
                #0  0x00007feffad4ce6d syscall (libc.so.6 + 0x10ae6d)
                #1  0x00005568302d6299 _ZN11parking_lot7condvar7Condvar19wait_until_internal17h9b4ec543cacea21cE (rpm-ostree + 0x2b3299)
                #2  0x000055683070adc5 _ZN75_$LT$tokio..park..thread..CachedParkThread$u20$as$u20$tokio..park..Park$GT$4park17h9b8f5c5e374b943dE (rpm-ostree + 0x6e7dc5)
                #3  0x00005568303fe62c _ZN5tokio4park6thread16CachedParkThread8block_on17hcf625c44b7a1db7fE (rpm-ostree + 0x3db62c)
                #4  0x00005568304a0664 _ZN5tokio7runtime11thread_pool10ThreadPool8block_on17h4f099aa5d392ff81E (rpm-ostree + 0x47d664)
                #5  0x0000556830439810 _ZN5tokio7runtime7Runtime8block_on17h9f358ab8990b8493E (rpm-ostree + 0x416810)
                #6  0x000055683047a409 _ZN10rpm_ostree4main17h6f3142cc3a940fc5E (rpm-ostree + 0x457409)
                #7  0x00005568304523e3 _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17hb6abc3c91df33356E (rpm-ostree + 0x42f3e3)
                #8  0x000055683049c2e9 _ZN3std2rt10lang_start28_$u7b$$u7b$closure$u7d$$u7d$17h3dd697f0760e63faE.llvm.13258593817370844734 (rpm-ostree + 0x4792e9)
                #9  0x00005568306cd859 _ZN3std2rt19lang_start_internal17hcbc5d1079a591413E (rpm-ostree + 0x6aa859)
                #10 0x000055683047ade8 main (rpm-ostree + 0x457de8)
                #11 0x00007feffac6f560 __libc_start_call_main (libc.so.6 + 0x2d560)
                #12 0x00007feffac6f60c __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2d60c)
                #13 0x0000556830165f55 _start (rpm-ostree + 0x142f55)

                Stack trace of thread 6202:
                #0  0x00007feffad4ce6d syscall (libc.so.6 + 0x10ae6d)
                #1  0x00007feffb82e90c g_cond_wait_until (libglib-2.0.so.0 + 0xa590c)
                #2  0x00007feffb7ae071 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0 + 0x25071)
                #3  0x00007feffb7ae1f6 g_async_queue_timeout_pop (libglib-2.0.so.0 + 0x251f6)
                #4  0x00007feffb80b1b9 g_thread_pool_thread_proxy.lto_priv.0 (libglib-2.0.so.0 + 0x821b9)
                #5  0x00007feffb808842 g_thread_proxy (libglib-2.0.so.0 + 0x7f842)
                #6  0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #7  0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5206:
                #0  0x00007feffad4ce6d syscall (libc.so.6 + 0x10ae6d)
                #1  0x00005568302d6299 _ZN11parking_lot7condvar7Condvar19wait_until_internal17h9b4ec543cacea21cE (rpm-ostree + 0x2b3299)
                #2  0x00005568306fef75 _ZN66_$LT$tokio..runtime..park..Parker$u20$as$u20$tokio..park..Park$GT$4park17h611b7986a2445208E (rpm-ostree + 0x6dbf75)
                #3  0x000055683070f119 _ZN5tokio7runtime11thread_pool6worker7Context12park_timeout17hd2aba060b318eeacE (rpm-ostree + 0x6ec119)
                #4  0x000055683070e662 _ZN5tokio7runtime11thread_pool6worker7Context3run17h3d56ab806a126f04E (rpm-ostree + 0x6eb662)
                #5  0x000055683070d746 _ZN5tokio7runtime11thread_pool6worker3run17h909af045f2300022E (rpm-ostree + 0x6ea746)
                #6  0x0000556830706200 _ZN5tokio7runtime4task7harness20Harness$LT$T$C$S$GT$4poll17hf762ac8bbe8eaec8E (rpm-ostree + 0x6e3200)
                #7  0x00005568307091c3 _ZN5tokio7runtime8blocking4pool5Inner3run17h991963d9f264a5e7E (rpm-ostree + 0x6e61c3)
                #8  0x00005568306fa74f _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h6f737ab7b0902fdaE (rpm-ostree + 0x6d774f)
                #9  0x000055683070bef4 _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17he4ff7ac48016b426E (rpm-ostree + 0x6e8ef4)
                #10 0x00005568306d6515 _ZN3std3sys4unix6thread6Thread3new12thread_start17hd3aa83834b83bd4aE (rpm-ostree + 0x6b3515)
                #11 0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #12 0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)

                Stack trace of thread 5210:
                #0  0x00007feffad4773f __poll (libc.so.6 + 0x10573f)
                #1  0x00007feffb83319c g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0xaa19c)
                #2  0x00007feffb7dd853 g_main_loop_run (libglib-2.0.so.0 + 0x54853)
                #3  0x00007feffba33c0a gdbus_shared_thread_func.lto_priv.0 (libgio-2.0.so.0 + 0x113c0a)
                #4  0x00007feffb808842 g_thread_proxy (libglib-2.0.so.0 + 0x7f842)
                #5  0x00007feffaccfa87 start_thread (libc.so.6 + 0x8da87)
                #6  0x00007feffad54640 __clone3 (libc.so.6 + 0x112640)
lucab commented 2 years ago

Just to make sure, this is happening on v2022.1, right?

I think this is some fallout from https://github.com/coreos/rpm-ostree/pull/3298, which landed in that version. From the stacktrace, this seems to be hitting an assertion, and I added a couple of new assertions in that PR. Either the invariants I'm checking are wrong, or there is some other fishy logic somewhere else.

Can you please attach the last few lines from the service journal too? If my reading is correct, the daemon should have logged an assertion failure right before coredumping.

lucab commented 2 years ago

Looking back at my PR, I saw at least one bug that I introduced, https://github.com/coreos/rpm-ostree/pull/3346 takes care of that. If the the failing assertion is the one on local_error != NULL, then that should take care of it.

travier commented 2 years ago

Logs:

Jan 19 14:48:43 fedora systemd[1]: Starting rpm-ostree System Management Daemon...
Jan 19 14:48:43 fedora rpm-ostree[3856]: Reading config file '/etc/rpm-ostreed.conf'
Jan 19 14:48:45 fedora rpm-ostree[3856]: In idle state; will auto-exit in 61 seconds
Jan 19 14:48:45 fedora systemd[1]: Started rpm-ostree System Management Daemon.
Jan 19 14:48:45 fedora rpm-ostree[3856]: client(id:cli dbus:1.135 unit:session-6.scope uid:0) added; new total=1
Jan 19 14:48:47 fedora rpm-ostree[3856]: Locked sysroot
Jan 19 14:48:47 fedora rpm-ostree[3856]: Initiated txn RefreshMd for client(id:cli dbus:1.135 unit:session-6.scope uid:0): /org/projectatomic/rpmostree1/fedora
Jan 19 14:48:47 fedora rpm-ostree[3856]: Process [pid: 3849 uid: 0 unit: session-6.scope] connected to transaction progress
Jan 19 14:48:47 fedora rpm-ostree[3856]: Librepo version: 1.14.2 with CURL_GLOBAL_ACK_EINTR support (libcurl/7.79.1 OpenSSL/1.1.1l-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.45.1 OpenLDAP/2.4.59)
Jan 19 14:48:48 fedora rpm-ostree[3856]: Txn RefreshMd on /org/projectatomic/rpmostree1/fedora successful
Jan 19 14:48:48 fedora rpm-ostree[3856]: Unlocked sysroot
Jan 19 14:48:48 fedora rpm-ostree[3856]: Process [pid: 3849 uid: 0 unit: session-6.scope] disconnected from transaction progress
Jan 19 14:48:49 fedora rpm-ostree[3856]: Enabled rpm-md repositories: fedora-cisco-openh264 updates fedora copr:copr.fedorainfracloud.org:marcdeop:frameworks copr:copr.fedorainfracloud.org:marcdeop:plasma updates-archive
Jan 19 14:48:50 fedora rpm-ostree[3856]: Importing rpm-md...done
Jan 19 14:48:50 fedora rpm-ostree[3856]: rpm-md repo 'fedora-cisco-openh264' (cached); generated: 2021-09-21T18:07:30Z solvables: 4
Jan 19 14:48:50 fedora rpm-ostree[3856]: rpm-md repo 'updates' (cached); generated: 2022-01-19T01:42:17Z solvables: 14915
Jan 19 14:48:50 fedora rpm-ostree[3856]: rpm-md repo 'fedora' (cached); generated: 2021-10-26T05:31:27Z solvables: 65732
Jan 19 14:48:50 fedora rpm-ostree[3856]: rpm-md repo 'copr:copr.fedorainfracloud.org:marcdeop:frameworks' (cached); generated: 2022-01-04T18:59:37Z solvables: 446
Jan 19 14:48:50 fedora rpm-ostree[3856]: rpm-md repo 'copr:copr.fedorainfracloud.org:marcdeop:plasma' (cached); generated: 2022-01-19T11:53:05Z solvables: 588
Jan 19 14:48:50 fedora rpm-ostree[3856]: rpm-md repo 'updates-archive' (cached); generated: 2022-01-19T02:49:26Z solvables: 21161
Jan 19 14:48:51 fedora rpm-ostree[3856]: **
Jan 19 14:48:51 fedora rpm-ostree[3856]: rpm-ostreed:ERROR:src/daemon/rpmostreed-os-experimental.cxx:292:gboolean osexperimental_handle_download_packages(RPMOSTreeOSExperimental*, GDBusMethodInvocation*, GUnixFDList*, const gchar* const*, const char*): assertion failed: (local_error != NULL)
Jan 19 14:48:51 fedora rpm-ostree[3856]: Bail out! rpm-ostreed:ERROR:src/daemon/rpmostreed-os-experimental.cxx:292:gboolean osexperimental_handle_download_packages(RPMOSTreeOSExperimental*, GDBusMethodInvocation*, GUnixFDList*, const gchar* const*, const char*): assertion failed: (local_error != NULL)
travier commented 2 years ago

I can confirm that this feature works with a lot of packages 🙂:

$ rpm-ostree status
State: idle
Deployments:
  fedora:fedora/35/x86_64/kinoite
                   Version: 35.20220119.0 (2022-01-19T01:37:55Z)
                BaseCommit: 13754a8e785971df267e031d075943a4d8eb96727e2cdc1b1951e054a940811d
              GPGSignature: Valid signature by 787EA6AE1147EEE56C40B30CDB4639719867C58F
                      Diff: 153 upgraded, 3 added
      ReplacedBasePackages: kf5-kdeclarative kf5-networkmanager-qt kf5-kpackage kf5-kiconthemes kf5-kdoctools kf5-kio-doc kf5-kinit kf5-kparts kf5-kbookmarks kf5-kwidgetsaddons kf5-kjobwidgets kf5-kpeople kf5-kidletime kf5-threadweaver kf5-kio-file-widgets kf5-sonnet-core kf5-frameworkintegration-libs kf5-kservice kf5-kunitconversion kf5-kconfig-gui kf5-kio-ntlm kf5-kwallet-libs kf5-kactivities kf5-kitemviews kf5-kjs kf5-baloo-file kf5-kcompletion kf5-filesystem kf5-modemmanager-qt kf5-bluez-qt kf5-kcrash kf5-kirigami2 kf5-kitemmodels kf5-kcoreaddons kf5-syntax-highlighting kf5-ktexteditor kf5-khtml kf5-knotifyconfig kf5-kdelibs4support kf5-kconfigwidgets kf5-frameworkintegration qqc2-desktop-style kf5-baloo-libs kf5-kcmutils kf5-kauth kf5-kdesu kf5-kxmlrpcclient kf5-kimageformats kf5-kpty kf5-attica kf5-kwindowsystem kf5-kwallet kf5-kglobalaccel-libs kf5-kio-gui kf5-kconfig-core kf5-kdnssd kf5-kwayland kf5-kio-core kf5-purpose kf5-kactivities-stats kf5-kdbusaddons kf5-kguiaddons kf5-kcodecs kf5-prison kf5-baloo kf5-kded kf5-knotifications kf5-krunner kf5-kio-widgets-libs kf5-karchive kf5-kglobalaccel kf5-knewstuff kf5-sonnet-ui kf5-solid kf5-kxmlgui kf5-kdelibs4support-libs kf5-kquickcharts breeze-icon-theme kf5-kfilemetadata kf5-kio-core-libs kf5-ktextwidgets kf5-ki18n kf5-kio-widgets kf5-plasma 5.89.0-1.fc35 -> 5.90.0-1.fc35
                            kf5-kholidays 1:5.89.0-1.fc35 -> 1:5.90.0-1.fc35 kscreen kdesu 1:5.23.4-1.fc35 -> 1:5.23.5-1.fc35
                            plasma-discover-notifier libksysguard plasma-discover-libs plasma-disks plasma-breeze libkworkspace5 kscreenlocker plasma-browser-integration kwin-libs plasma-vault pam-kwallet kactivitymanagerd breeze-gtk-common plasma-nm-openvpn plasma-desktop plasma-discover-flatpak kde-gtk-config plasma-workspace-geolocation-libs xdg-desktop-portal-kde plasma-workspace-common plasma-workspace-wayland plasma-drkonqi plasma-nm-openconnect kmenuedit bluedevil khotkeys plasma-workspace-libs plasma-nm oxygen-sound-theme kdeplasma-addons layer-shell-qt plasma-lookandfeel-fedora kwayland-integration sddm-kcm kwin plasma-systemmonitor plasma-discover sddm-breeze plasma-systemsettings plasma-workspace kde-cli-tools libksysguard-common kwin-common breeze-gtk-gtk4 powerdevil plasma-milou plasma-breeze-common plasma-thunderbolt plasma-integration ksystemstats plasma-workspace-geolocation kwin-x11 plasma-desktop-doc kdecoration plasma-pa polkit-kde kwrited breeze-cursor-theme kinfocenter breeze-gtk-gtk3 libkscreen-qt5 kwayland-server plasma-workspace-x11 kwin-wayland plasma-nm-vpnc 5.23.4-1.fc35 -> 5.23.5-1.fc35
                            rpm-ostree-libs rpm-ostree 2022.1-2.fc35 -> 2022.1.40.g66d86114-1.fc35
             LocalPackages: rpm-ostree-debuginfo-2022.1.40.g66d86114-1.fc35.x86_64 rpm-ostree-libs-debuginfo-2022.1.40.g66d86114-1.fc35.x86_64