BUG: Crash while getting problem with file dedup

[nefelim4ag]

Hi, i've get a crash while dedup fs several times, with several errors, Every crash looks like:

[0x3692190] Add extent for file "/mnt/@arch/usr/lib/wine/d3dx9_36.dll.so" at offset 328.0K (7)
[0x3692190] Dedupe 1 extents (id: f5fb7835) with target: (328.0K, 4.0K), "/mnt/@arch/usr/lib/wine/d3dx9_37.dll.so"
Error 26: Text file busy while opening "/mnt/@arch/usr/share/google-play-music-desktop-player/Google Play Music Desktop Player" (write=1)
/mnt/@arch/usr/share/google-play-music-desktop-player/Google Play Music Desktop Player: Skipping dedupe.
[0x36921e0] Add extent for file "/mnt/@arch/usr/share/skypeforlinux/skypeforlinux" at offset 50.8M (6)
ERROR: run_dedupe.c:399
[stack trace follows]
duperemove(print_stack_trace+0x2b) [0x408c3b]
duperemove() [0x40b3a1]
duperemove() [0x40b8a3]
/usr/lib/libglib-2.0.so.0(+0x70cae) [0x7fd547e3acae]
/usr/lib/libglib-2.0.so.0(+0x702b5) [0x7fd547e3a2b5]
/usr/lib/libpthread.so.0(+0x7484) [0x7fd546f8e484]
/usr/lib/libc.so.6(clone+0x6d) [0x7fd5474fe6dd]

But, sorry, at now, i can't create a reproducible test.

[markfasheh]

NP about the non-reproducer. The line causing it is this one in dedupe_extent_list():

abort_on(ctxt != NULL);

So it seems we're leaking a dedupe context in some cases. And actually the output before the crash shows we got at least one error opening so I'll check that part out first, thanks.

[markfasheh]

Can you try the patch in the 'issue#132' branch ( https://github.com/markfasheh/duperemove/tree/issue%23132 ):

0c41956acda413ee0b25dda8d26654f299930d71 fixes what I believe to be the cause of your crash. I couldn't reproduce the conditions here on my own so I relied on a read-through of the code to spot the issue.

[markfasheh]

Ok I was able to come up with a reproducer (pasted below if you happen to be interested). The patch fixes this for me.

DEST=/btrfs/ DUPEREMOVE=/build/mfasheh/duperemove.git/duperemove TMPUSER=mfasheh BS=$((128*1024))

rm -f $DEST/1 $DEST/2 dd if=/dev/zero of=$DEST/1 bs=$BS count=1 dd if=/dev/zero of=$DEST/2 bs=$BS count=1 chown $TMPUSER 2 sudo -u $TMPUSER $DUPEREMOVE -vd 1 2

[nefelim4ag]

@markfasheh, i've reproduce my test case, and you branch issue#132 fix this crash, thanks.

---

P.S. Can't reproduce problem with you script, before and after fixes. Update: Sorry, my error, some problem reproduced and fixed.

$ /tmp/test_no_fix.sh     
1+0 records in
1+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.000172069 s, 762 MB/s
1+0 records in
1+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.000152793 s, 858 MB/s
Using 128K blocks
Using hash: murmur3

$ /tmp/test_132.sh 
1+0 records in
1+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.000294788 s, 445 MB/s
1+0 records in
1+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.000227996 s, 575 MB/s
Using 128K blocks
Using hash: murmur3
Error 2: No such file or directory while getting path to file 1. Skipping.
Error 2: No such file or directory while getting path to file 2. Skipping.

P.S.S. With fixes i find new messages: "...had status (-22) "Invalid argument" I think it's expected.

Thanks you!

[markfasheh]

oh my script was sligthly broken (but works on my machine because i happened to be in the correct directory). For the record, this actually works regardless of whether you're in the test directory:

DEST=/btrfs/ DUPEREMOVE=/build/mfasheh/duperemove.git/duperemove TMPUSER=mfasheh BS=$((128*1024))

rm -f $DEST/1 $DEST/2 dd if=/dev/zero of=$DEST/1 bs=$BS count=1 dd if=/dev/zero of=$DEST/2 bs=$BS count=1 chown $TMPUSER $DEST/2 sudo -u $TMPUSER $DUPEREMOVE -vd $DEST/1 $DEST/2

Anyway thanks for testing this fix is in master now.