My weekly backups have started failing and i have narrowed it down to the planner crashing due to me removing a disklistentry.
I was running amanda 3.4.x but also tried updating to 3.5.1 arch linux. Also tried newer gnulib (the one in pull request 211).
Example of segfaulting command:
/usr/lib/amanda/planner backup --starttime 20230730134014 --log-filename /tmp/amlog --exact-match
The cause seems to be that i removed a disklist entry and the planner is now still attempting to backup the old entry but gets a null program name to use for it.
If i have only 1 disklist entry (any one of the ones i have) it does not segfault, but as soon as i get two it segfaults.
With the 1 entry though it does not actually do any estimate it just returns: planner: find_est_for_dp return NULL.
I found in my old notes that i should have done "sudo -u amanda amadmin backup delete localhost test" and then removed it from the disklist. But I feel like the planner should not be crashing at all, and instead print an error if it thinks my config is invalid.
Crash details:
# gdb /usr/lib/amanda/planner
(gdb) start backup --starttime 20230730134014 --log-filename /tmp/amlog --exact-match
...
(gdb) c
Continuing.
planner: pid 2634079 executable /usr/lib/amanda/planner version 3.5.4.git.ac850eb5
planner: build: VERSION="Amanda-3.5.4.git.ac850eb5" BUILT_DATE=""
planner: BUILT_MACH="" BUILT_REV=UNDEF BUILT_BRANCH=UNDEF CC="gcc"
planner: paths: bindir="/usr/bin" sbindir="/usr/bin"
planner: libexecdir="/usr/lib" amlibexecdir="/usr/lib/amanda"
planner: mandir="/usr/share/man"
planner: AMANDA_TMPDIR="/tmp/amandabackup-2561793"
planner: AMANDA_DBGDIR="/tmp/amandabackup-2561793"
planner: CONFIG_DIR="/etc/amanda" DEV_PREFIX="/dev/" DUMP=UNDEF
planner: RESTORE=UNDEF VDUMP=UNDEF VRESTORE=UNDEF XFSDUMP=UNDEF
planner: XFSRESTORE=UNDEF VXDUMP=UNDEF VXRESTORE=UNDEF
planner: SAMBA_CLIENT="/usr/bin/smbclient" GNUTAR="/usr/bin/tar"
planner: COMPRESS_PATH="/usr/bin/gzip"
planner: UNCOMPRESS_PATH="/usr/bin/gzip" LPR="/usr/bin/lpr"
planner: MAILER="/usr/bin/mailx"
planner: listed_incr_dir="/var/lib/amanda/gnutar-lists"
planner: defs: DEFAULT_SERVER="minerva" DEFAULT_CONFIG="DailySet1"
planner: DEFAULT_TAPE_SERVER="minerva" DEFAULT_TAPE_DEVICE=""
planner: NEED_STRSTR AMFLOCK_POSIX AMFLOCK_FLOCK AMFLOCK_LOCKF
planner: AMFLOCK_LNLOCK SETPGRP_VOID AMANDA_DEBUG_DAYS=4 BSD_SECURITY
planner: USE_AMANDAHOSTS CLIENT_LOGIN="amanda" CHECK_USERID HAVE_GZIP
planner: COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
planner: COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
planner: CONFIGURE_ARGS=" '--prefix=/usr' '--sbindir=/usr/bin' '--libexecdir=/usr/lib' '--sysconfdir=/etc' '--localstatedir=/var' '--with-configdir=/etc/amanda' '--with-security-file=/etc/amanda-security.conf' '--with-gnutar-listdir=/var/lib/amanda/gnutar-lists' '--enable-manpage-build' '--mandir=/usr/share/man' '--with-user=amanda' '--with-group=amanda' '--with-ipv6' '--with-ssh-security' '--with-amandates=/var/lib/amanda/amandates' '--with-tmpdir=/tmp/amandabackup-2561793' 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-stack-check -g -rdynamic -fcommon' 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'"
READING CONF INFO...
DATE 20230730134014
planner: timestamp 20230730134014
planner: tape_length is set from tape length (1468006400 KB) * runtapes (3) == 4404019200 KB
planner: time 0.000: startup took 0.000 secs
SENDING FLUSHES...
Cleaning up holding disk '/dumps'
ENDFLUSH
ENDFLUSH
SETTING UP FOR ESTIMATES...
planner: time 0.001: setting up estimates for localhost:efi
setup_estimate: localhost:efi: command 0, options: none last_level 0 next_level0 5 level_days 3 getting estimates 0 (-3) 1 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:boot
setup_estimate: localhost:boot: command 0, options: none last_level 0 next_level0 5 level_days 77 getting estimates 0 (-3) 1 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:root
localhost:root overdue 8 days for level 0
setup_estimate: localhost:root: command 0, options: none last_level 1 next_level0 -8 level_days 2 getting estimates 0 (-3) 1 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:storage
localhost:storage overdue 22 days for level 0
setup_estimate: localhost:storage: command 0, options: none last_level 2 next_level0 -22 level_days 1 getting estimates 0 (-3) 2 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:zfs-priv
localhost:zfs-priv overdue 19576 days for level 0
setup_estimate: localhost:zfs-priv: command 0, options: none last_level -1 next_level0 -19576 level_days 0 getting estimates 0 (-3) -1 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:exuvo-desktop-root
localhost:exuvo-desktop-root overdue 14 days for level 0
setup_estimate: localhost:exuvo-desktop-root: command 0, options: none last_level 2 next_level0 -14 level_days 1 getting estimates 0 (-3) 2 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:backup
setup_estimate: localhost:backup: command 0, options: none last_level 0 next_level0 47 level_days 42 getting estimates 0 (-3) 1 (-3) -1 (-3)
planner: time 0.001: setting up estimates for localhost:backup-minerva
localhost:backup-minerva overdue 23 days for level 0
setup_estimate: localhost:backup-minerva: command 0, options: none last_level 0 next_level0 -23 level_days 2 getting estimates 1 (-3) -1 (-3) -1 (-3)
planner: time 0.002: setting up estimates took 0.001 secs
GETTING ESTIMATES...
[Detaching after fork from child process 2634086]
[Detaching after fork from child process 2634094]
[New Thread 0x7ffff6c9e6c0 (LWP 2634103)]
[Detaching after fork from child process 2634104]
[Thread 0x7ffff6c9e6c0 (LWP 2634103) exited]
Thread 1 "planner" received signal SIGSEGV, Segmentation fault.
#0 0x00007ffff7b5ff6b in () at /usr/lib/libc.so.6
#1 0x000055555555f52a in client_estimate_as_xml (nr_estimates=1, features=<optimized out>, ep=0x5555555b3120) at planner.c:1672
#2 getsize (hostp=0x5555555940b0) at planner.c:1981
#3 0x00005555555610fd in getsize (hostp=0x5555555940b0) at planner.c:1825
#4 handle_result (datap=0x5555555940b0, pkt=0x7fffffffc5f0, sech=0x7ffff0000bb0) at planner.c:2415
#5 0x00007ffff7f1b5d1 in state_machine (p=0x5555555b0540, action=PA_RCVDATA, pkt=0x7fffffffc5f0) at protocol.c:553
#6 0x00007ffff7f21c20 in recvpkt_callback (cookie=0x7ffff0000bb0, buf=<optimized out>, bufsize=<optimized out>) at security-util.c:2245
#7 0x00007ffff7f1e48b in stream_read_callback (arg=0x7ffff00028e0) at security-util.c:2354
#8 0x00007ffff7f21db1 in sec_tcp_conn_read_callback (cookie=0x7ffff0001840) at security-util.c:2490
#9 0x00007ffff7f129ff in event_handle_callback (user_ptr=0x5555555b32e0) at event.c:118
#10 event_handle_callback (user_ptr=0x5555555b32e0) at event.c:110
#11 0x00007ffff7db8a31 in g_main_context_dispatch () at /usr/lib/libglib-2.0.so.0
#12 0x00007ffff7e15cc9 in () at /usr/lib/libglib-2.0.so.0
#13 0x00007ffff7db60e2 in g_main_context_iteration () at /usr/lib/libglib-2.0.so.0
#14 0x00007ffff7f1295c in event_loop_wait (wait_eh=0x0, nonblock=0, return_when_empty=1) at event.c:427
#15 0x00007ffff7f12ed2 in event_loop (nonblock=<optimized out>) at event.c:328
#16 0x00007ffff7f1bd89 in protocol_run () at protocol.c:421
#17 0x0000555555559005 in get_estimates () at planner.c:1601
#18 main (argc=<optimized out>, argv=<optimized out>) at planner.c:694
My weekly backups have started failing and i have narrowed it down to the planner crashing due to me removing a disklistentry. I was running amanda 3.4.x but also tried updating to 3.5.1 arch linux. Also tried newer gnulib (the one in pull request 211). Example of segfaulting command: /usr/lib/amanda/planner backup --starttime 20230730134014 --log-filename /tmp/amlog --exact-match
The cause seems to be that i removed a disklist entry and the planner is now still attempting to backup the old entry but gets a null program name to use for it. If i have only 1 disklist entry (any one of the ones i have) it does not segfault, but as soon as i get two it segfaults. With the 1 entry though it does not actually do any estimate it just returns: planner: find_est_for_dp return NULL.
I found in my old notes that i should have done "sudo -u amanda amadmin backup delete localhost test" and then removed it from the disklist. But I feel like the planner should not be crashing at all, and instead print an error if it thinks my config is invalid.
Crash details: