rr-debugger / rr

Record and Replay Framework
http://rr-project.org/
Other
9.1k stars 580 forks source link

Assertion while recording `ret == -ENOENT || ret == -ENODEV || ret == -ENOTBLK || ret == -EINVAL' failed to hold. unknown quotactl(0x5805) #2013

Open aaptel opened 7 years ago

aaptel commented 7 years ago

Me again unfortunately with another smbd issue. Recording fails with this assertion failure (using latest git 1022e348 ):

[FATAL /home/aaptel/prog/rr/rr.git/src/record_syscall.cc:5009:rec_process_syscall_arch() errno: SUCCESS] 
 (task 28255 (rec:28255) at time 61003)
 -> Assertion `ret == -ENOENT || ret == -ENODEV || ret == -ENOTBLK || ret == -EINVAL' failed to hold.  unknown quotactl(0x5805)
Launch gdb with
  gdb '-l' '10000' '-ex' 'target extended-remote :28255' /tmp/smb/sbin/smbd

Steps to reproduce:

git clone git://git.samba.org/samba.git samba
cd samba
./buildtools/bin/waf configure -C -j8 \
                                    --disable-cups --enable-selftest --with-libarchive  \
                                    --enable-debug --picky-developer \
                                    --enable-developer \
                                    --bundled-libraries=ALL \
                                    --prefix=/tmp/smb/ \
                                    --localstatedir=/tmp/smbstate
./buildtools/bin/waf build -j8
./buildtools/bin/waf install
rm -rf /tmp/share
mkdir -p /tmp/share
chmod 777 /tmp/share
cat <<EOF > /tmp/smb/etc/smb.conf
[global]
logging = ringbuf
log level = 0

[share]
path = /tmp/share
read only = no
guest ok = yes
EOF
for i in $(seq 100 9000); do
    echo ABC${i}DEF > /tmp/share/${i}xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
done

as root:

rr record /tmp/smb/sbin/smbd -i

in other shell:

/tmp/smb/bin/smbclient //localhost/share -U% -c ls
khuey commented 7 years ago

Can you attach gdb (using the command given) and see what quotactl(2) subcommand samba is making here?

aaptel commented 7 years ago

Unfortunately I always have this problem when I attach to smbd, stack seems corrupted:

root /home/aaptel/prog/bugs/bsc1028820 # gdb -q '-l' '10000' '-ex' 'target extended-remote :6349' /tmp/smb/sbin/smbd
Reading symbols from /tmp/smb/sbin/smbd...done.
Remote debugging using :6349
warning: remote target does not support file transfer, attempting to access files from local filesystem.
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug/lib64/ld-2.22.so.debug...done.
done.
0x0000000070000002 in _start () from /lib64/ld-linux-x86-64.so.2
(gdb) bt
#0  0x0000000070000002 in _start () from /lib64/ld-linux-x86-64.so.2
#1  0x00007fae7282aac3 in ?? ()
#2  0x0000000000000000 in ?? ()
(gdb) 
aaptel commented 7 years ago

I've debbuged smbd manually. /tmp is on an xfs filesystem on my desktop.

quotactl(QCMD(Q_XGETQSTAT,USRQUOTA), bdev,       -1, (caddr_t)&F);
// thats       5768448              "/dev/sda1"

F is a zeroed-out struct fs_quota_stat F;

aaptel commented 7 years ago

I don't know why I didn't think of this earlier but here's the strace output of the whole smbd session

root /home/aaptel/prog/bugs/bsc1028820 # strace -e quotactl  /tmp/smb/sbin/smbd -i
smbd version 4.7.0pre1-GIT-782172a9bef started.
Copyright Andrew Tridgell and the Samba Team 1992-2017
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=16158, si_uid=0, si_status=0, si_utime=0, si_stime=0} ---
quotactl(Q_XGETQSTAT|USRQUOTA, "/dev/sda1", 0xffffffff, 0x7ffc9c053e10) = -1 ENOSYS (Function not implemented)
quotactl(Q_XGETQSTAT|GRPQUOTA, "/dev/sda1", 0xffffffff, 0x7ffc9c053e10) = -1 ENOSYS (Function not implemented)
--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=16157, si_uid=0} ---
+++ killed by SIGTERM +++
khuey commented 7 years ago

We don't support Q_XGETQSTAT yet. It should be fairly straightforward to add. At https://github.com/mozilla/rr/blob/1022e348a4a90803eb88503d45389756a2076629/src/record_syscall.cc#L3620 we need to handle Q_XGETQSTAT by registering the fs_quota_stat struct outparam. At https://github.com/mozilla/rr/blob/1022e348a4a90803eb88503d45389756a2076629/src/record_syscall.cc#L4995 we need to not assert because it's unhandled (which is what's happening now), and then we need to add Q_XGETQSTAT to the quotactl test.

You want to give it a try? :)