Neomutt crashes when reading INBOX folder with std::bad_alloc IOT instruction (core dumped)

gerazov commented 1 year ago

Expected Behaviour

Neomutt opens folder.

Actual Behaviour

Neomutt crashes.

If NeoMutt crashed, did your OS create a 'coredump' file?

Yes.

Steps to Reproduce

The problem started when syncing the account with isync via mutt-wizard and the hard drive filled up. When trying to start neomutt it now crashes.

My gmail folder is opened fine and the same e-mail works fine on my laptop.

I've tried deleting and resyncing the folder which works, but again opening the folder crashes neomutt with the message:

Reading ~/.local/share/mail/gerazov@feit.ukim.edu.mk/INBOX... 0/156 (0%)terminate called after throwing an instance of 'std::bad_alloc'
       what():  std::bad_alloc
                              [1]    69664 IOT instruction (core dumped)  neomutt
                                                                                 %

I've followed https://github.com/neomutt/neomutt/issues/3106 to analyze the coredump and get:

$ gdb /usr/bin/neomutt neomutt_core_dump
GNU gdb (GDB) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/neomutt...

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.archlinux.org>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
Downloading separate debug info for /usr/bin/neomutt
(No debugging symbols found in /usr/bin/neomutt)
[New LWP 55781]
[New LWP 55782]
Downloading separate debug info for /usr/lib/libgssapi_krb5.so.2
Downloading separate debug info for /usr/lib/libgdbm.so.6
Downloading separate debug info for /usr/lib/libncursesw.so.6
Downloading separate debug info for /usr/lib/libz.so.1
Downloading separate debug info for /usr/lib/libsqlite3.so.0
Downloading separate debug info for /usr/lib/libnotmuch.so.5
Downloading separate debug info for /usr/lib/libgpgme.so.11
Downloading separate debug info for /usr/lib/libsasl2.so.3
Downloading separate debug info for /usr/lib/liblua5.3.so.5.3
Downloading separate debug info for /usr/lib/libgnutls.so.30
Downloading separate debug info for /usr/lib/libidn2.so.0
Downloading separate debug info for /usr/lib/liblmdb.so
Downloading separate debug info for /usr/lib/libkyotocabinet.so.16
Downloading separate debug info for /usr/lib/liblz4.so.1
Downloading separate debug info for /usr/lib/libzstd.so.1
Downloading separate debug info for /usr/lib/libc.so.6
Downloading separate debug info for /usr/lib/libkrb5.so.3
--Type <RET> for more, q to quit, c to continue without paging--
Downloading separate debug info for /usr/lib/libk5crypto.so.3
Downloading separate debug info for /usr/lib/libcom_err.so.2
Downloading separate debug info for /usr/lib/libkrb5support.so.0
Downloading separate debug info for /usr/lib/libkeyutils.so.1
Downloading separate debug info for /usr/lib/libresolv.so.2
Downloading separate debug info for /lib64/ld-linux-x86-64.so.2
Downloading separate debug info for /usr/lib/libm.so.6
Downloading separate debug info for /usr/lib/libgmime-3.0.so.0
Downloading separate debug info for /usr/lib/libgobject-2.0.so.0
Downloading separate debug info for /usr/lib/libglib-2.0.so.0
Downloading separate debug info for /usr/lib/libtalloc.so.2
Downloading separate debug info for /usr/lib/libxapian.so.30
Downloading separate debug info for /usr/lib/libsexp.so.1
Downloading separate debug info for /usr/lib/libassuan.so.0
Downloading separate debug info for /usr/lib/libgpg-error.so.0
Downloading separate debug info for /usr/lib/libdl.so.2
Downloading separate debug info for /usr/lib/libp11-kit.so.0
Downloading separate debug info for /usr/lib/libbrotlienc.so.1
Downloading separate debug info for /usr/lib/libbrotlidec.so.1
Downloading separate debug info for /usr/lib/libunistring.so.5
Downloading separate debug info for /usr/lib/libtasn1.so.6
Downloading separate debug info for /usr/lib/libnettle.so.8
Downloading separate debug info for /usr/lib/libhogweed.so.6
Downloading separate debug info for /usr/lib/libgmp.so.10
Downloading separate debug info for /usr/lib/libpthread.so.0
Downloading separate debug info for /usr/lib/libgio-2.0.so.0
Downloading separate debug info for /usr/lib/libffi.so.8
Downloading separate debug info for /usr/lib/libpcre2-8.so.0
Downloading separate debug info for /usr/lib/libuuid.so.1
                                                                                                                                           --Type <RET> for more, q to quit, c to continue without paging--
Downloading separate debug info for /usr/lib/libbrotlicommon.so.1
Downloading separate debug info for /usr/lib/libgmodule-2.0.so.0
Downloading separate debug info for /usr/lib/libmount.so.1
Downloading separate debug info for /usr/lib/libblkid.so.1
Downloading separate debug info for /usr/lib/gconv/ISO8859-1.so
Downloading separate debug info for /usr/lib/libnss_mymachines.so.2
Downloading separate debug info for /usr/lib/libcap.so.2
Downloading separate debug info for /usr/lib/libnss_mdns_minimal.so.2
Downloading separate debug info for /usr/lib/libnss_resolve.so.2
Downloading separate debug info for system-supplied DSO at 0x7ffed4d29000
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `neomutt'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f8e036a08ec in ?? () from /usr/lib/libc.so.6
[Current thread is 1 (Thread 0x7f8e025fca00 (LWP 55781))]
(gdb) bt
#0  0x00007f8e036a08ec in  () at /usr/lib/libc.so.6
#1  0x00007f8e03651ea8 in raise () at /usr/lib/libc.so.6
#2  0x00007f8e0363b53d in abort () at /usr/lib/libc.so.6
#3  0x00007f8e02e9a833 in __gnu_cxx::__verbose_terminate_handler() () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
#4  0x00007f8e02ea6d0c in __cxxabiv1::__terminate(void (*)()) (handler=<optimized out>)
    at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:48
#5  0x00007f8e02ea6d79 in std::terminate() () at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:58
#6  0x00007f8e02ea6fdd in __cxxabiv1::__cxa_throw(void*, std::type_info*, void (*)(void*))
    (obj=<optimized out>, tinfo=0x7f8e03028db8 <typeinfo for std::bad_alloc>, dest=0x7f8e02ea52b0 <std::bad_alloc::~bad_alloc()>)
    at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/eh_throw.cc:98
#7  0x00007f8e02e9a3fb in operator new(unsigned long) (sz=15020984193530737217)
    at /usr/src/debug/gcc/gcc/libstdc++-v3/libsupc++/new_op.cc:54
#8  0x00007f8e04122c73 in kyotocabinet::HashDB::read_record_body(kyotocabinet::HashDB::Record*) () at /usr/lib/libkyotocabinet.so.16
#9  0x00007f8e04122bf5 in kyotocabinet::HashDB::read_record(kyotocabinet::HashDB::Record*, char*) () at /usr/lib/libkyotocabinet.so.16
#10 0x00007f8e04120f08 in kyotocabinet::HashDB::copy_records(kyotocabinet::HashDB*) () at /usr/lib/libkyotocabinet.so.16
#11 0x00007f8e04120a96 in kyotocabinet::HashDB::reorganize_file(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () at /usr/lib/libkyotocabinet.so.16
#12 0x00007f8e04117dff in kyotocabinet::HashDB::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int) () at /usr/lib/libkyotocabinet.so.16
#13 0x00007f8e04149748 in kyotocabinet::PlantDB<kyotocabinet::HashDB, (unsigned char)49>::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int) () at /usr/lib/libkyotocabinet.so.16
#14 0x00007f8e0413ac15 in kyotocabinet::PolyDB::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int) () at /usr/lib/libkyotocabinet.so.16
#15 0x00007f8e040f5b90 in kcdbopen () at /usr/lib/libkyotocabinet.so.16
#16 0x000055cd61961bf5 in  ()
#17 0x000055cd61b734c6 in  ()
#18 0x000055cd61aafdfa in  ()
#19 0x000055cd61ab0ee0 in  ()
#20 0x000055cd61ab2e09 in  ()
#21 0x000055cd618a5970 in  ()
#22 0x000055cd61878860 in  ()
#23 0x00007f8e0363c790 in  () at /usr/lib/libc.so.6
#24 0x00007f8e0363c84a in __libc_start_main () at /usr/lib/libc.so.6
#25 0x000055cd61817045 in  ()
(gdb) quit

How often does this happen?

On a particular email

When did it start to happen?

When I synced the e-mail and filled up the hard-drive.

NeoMutt Version


~
NeoMutt 20230517-192-33461a-dirty
Copyright (C) 1996-2022 Michael R. Elkins and others.
NeoMutt comes with ABSOLUTELY NO WARRANTY; for details type 'neomutt -vv'.
NeoMutt is free software, and you are welcome to redistribute it
under certain conditions; type 'neomutt -vv' for details.

System: Linux 6.1.29-1-MANJARO (x86_64)
ncurses: ncurses 6.4.20221231 (compiled with 6.4.20221231)
libidn2: 2.3.4 (compiled with 2.3.4)
GPGME: 1.20.0
GnuTLS: 3.8.0
libnotmuch: 5.6.0
storage: kyotocabinet, gdbm, bdb, lmdb
compression: lz4, zlib, zstd

Configure options: --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib --gpgme --sqlite --autocrypt --lua --notmuch --gss --gnutls --sasl --with-idn2=/usr --disable-idn --idn2 --bdb --lmdb --kyotocabinet --gdbm --lz4 --zlib --zstd

Compilation CFLAGS: -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -fno-delete-null-pointer-checks -D_ALL_SOURCE=1 -D_GNU_SOURCE=1 -D__EXTENSIONS__ -D_XOPEN_SOURCE_EXTENDED -I/usr/include -I/usr/include -I/usr/include -DNCURSES_WIDECHAR -I/usr/include/p11-kit-1 -I/include -I/usr/include/db5.3 -I/usr/include -I/usr/include -O2

Default options:
  +attach_headers_color +compose_to_sender +compress +cond_date +debug
  +encrypt_to_self +forgotten_attachments +forwref +ifdef +imap +index_color
  +initials +limit_current_thread +multiple_fcc +nested_if +new_mail +nntp +pop
  +progress +quasi_delete +regcomp +reply_with_xorig +sensible_browser +sidebar
  +skip_quoted +smtp +status_color +timeout +tls_sni +trash

Compile options:
  +autocrypt +fcntl -flock -fmemopen +futimens +getaddrinfo +gnutls +gpgme
  -gsasl +gss +hcache -homespool +idn +inotify -locales_hack +lua -mixmaster
  +nls +notmuch -openssl +pgp +regex +sasl +smime +sqlite +sun_attachment

MAILPATH="/var/mail"
PKGDATADIR="/usr/share/neomutt"
SENDMAIL="/usr/sbin/sendmail"
SYSCONFDIR="/etc"

To learn more about NeoMutt, visit: https://neomutt.org
If you find a bug in NeoMutt, please raise an issue at:
    https://github.com/neomutt/neomutt/issues
or send an email to: <neomutt-devel@neomutt.org>

Extra Info

Operating System and its version Manjaro Linux 22.1.3
Were you using multiple copies of NeoMutt at once? No
Were you using 'screen' or 'tmux'? tmux
Is your email local (maildir) or remote (IMAP)? IMAP

flatcap commented 1 year ago

Hmm... Sorry about that. Thanks for the debugging.

It looks like the header cache is the problem. Try deleting it.

I notice that your NeoMutt has lmdb built-in. This is a modern database and much faster than kyotocabinet. You may also want to consider compression as this can halve the size of the files, e.g.

set header_cache_backend = "lmdb"
set header_cache_compress_level = 18
set header_cache_compress_method = "zstd"

Explanation...

The header cache code stores a binary representation of the structures in memory. This is fast, but risky. If the structures change, bad things happen. We do a LOT of checks to make sure that the on-disk structures matche those in-code, but sometimes it's not enough.

gerazov commented 1 year ago

Wow! thanks a million :pray: :pray: :pray:

I couldn't solve this for a while now :sweat_smile:

The header cache for me was in ~/.cache/mutt-wizzard instead of ~/.cache/mutt (https://wiki.archlinux.org/title/mutt)

And wow lmdb is soo much faster :zap: :zap: :zap: - awesome tip :sunglasses: - this should be default!

neomutt / neomutt