search

sahib / rmlint

Extremely fast tool to remove duplicates and other lint from your filesystem
http://rmlint.rtfd.org
GNU General Public License v3.0
1.86k stars 128 forks source link

rmlint develop crashes during preprocessing of a large directory tree #596

Open intelfx opened 1 year ago

intelfx commented 1 year ago

OS/Environment

Arch Linux x86_64, Linux 6.0.12, btrfs filesystem, rmlint 2.10.1.r281.g58d29ec1-1

Command line

rmlint -T df -c sh:handler=clone --hidden --xattr-clear --xattr-read --xattr-write --hash-unmatched -vv .

Log

DEBUG: Freeing device 0 (pointer 0x5555555eb400)
DEBUG: Found 3732428 files, ignored 0 hidden files and 0 hidden folders
DEBUG: List build finished at 46,066 with 3732428 files
DEBUG: rm_preprocessstarting at 46,066;
[Thread 0x7ffff73ff6c0 (LWP 216485) exited]
DEBUG: initial size sort finished at time 48,002; sorted 3732428 files
DEBUG: path doubles removal/hardlink bundling/other lint finished at 51,179; removed 95085 of 3732428
DEBUG: preparing size groups for shredding (dupe finding)...
g_path_get_dirname: assertion 'file_name != NULL' failed
Thread 1 "rmlint" received signal SIGSEGV, Segmentation fault.

Backtrace

(gdb) bt
#0  0x00007ffff7a2870b in  () at /usr/lib/libc.so.6
#1  0x00005555555785e1 in rm_mounts_get_disk_id (self=0x5555555d74d0, dev=1048690, path=path@entry=0x0) at lib/utilities.c:1094
#2  0x00005555555695c3 in rm_mds_device_get (mds=0x5555555b9b60, path=path@entry=0x0, dev=<optimized out>) at lib/md-scheduler.c:400
#3  0x000055555556f6e9 in rm_shred_disk (file=file@entry=0x7ffff018a920, session=session@entry=0x7fffffffda00) at lib/shredder.c:707
#4  0x0000555555570e03 in rm_shred_file_preprocess (file=0x7ffff018a920, group=group@entry=0x7fffffffd7c0) at lib/shredder.c:1107
#5  0x000055555557102e in rm_shred_process_group (files=<optimized out>, main=<optimized out>) at lib/shredder.c:1193
#6  0x00007ffff7c2ee90 in g_slist_foreach () at /usr/lib/libglib-2.0.so.0
#7  0x0000555555570bdb in rm_shred_preprocess_input (main=main@entry=0x7fffffffd870) at lib/shredder.c:1214
#8  0x000055555557165c in rm_shred_run (session=session@entry=0x7fffffffda00) at lib/shredder.c:1773
#9  0x000055555556511b in rm_cmd_main (session=session@entry=0x7fffffffda00) at lib/cmdline.c:1473
#10 0x000055555555e813 in main (argc=12, argv=0x7fffffffde58) at src/rmlint.c:134
cebtenzzre commented 1 year ago

This is a known issue with btrfs subvolumes that I have a fix for in commit https://github.com/Cebtenzzre/rmlint/commit/4a34fbd3acecbe7518b33c91e725cd10c69c369b. I believe it only affects develop, and at a glance it appears to be a regression caused by a575d6ef2646c1fac1abaa6df79efecaa88e02d9.