search

sahib / rmlint

Extremely fast tool to remove duplicates and other lint from your filesystem
http://rmlint.rtfd.org
GNU General Public License v3.0
1.87k stars 130 forks source link

Aborts after a few seconds with free(): invalid pointer #422

Closed austinbutler closed 4 years ago

austinbutler commented 4 years ago

I haven't run this since September 2019, but it did work fine then.

> $ rmlint -vvv -o csv:rmlint.csv /mnt/data
DEBUG: Filesystem /sys/fs/fuse/connections: not reflink capable
DEBUG: Filesystem /run/user/1000/gvfs: not reflink capable
DEBUG: Filesystem /run/user/1000: not reflink capable
DEBUG: Filesystem /var/lib/nfs/rpc_pipefs: not reflink capable
DEBUG: Filesystem /mnt/data: not reflink capable
DEBUG: Filesystem /boot: not reflink capable
DEBUG: Filesystem /proc/fs/nfsd: not reflink capable
DEBUG: `configfs` mount detected at /sys/kernel/config (#19); Ignoring all files in it.
DEBUG: Filesystem /sys/kernel/config: not reflink capable
DEBUG: Filesystem /tmp: not reflink capable
DEBUG: Filesystem /sys/kernel/tracing: not reflink capable
DEBUG: `debugfs` mount detected at /sys/kernel/debug (#6); Ignoring all files in it.
DEBUG: Filesystem /sys/kernel/debug: not reflink capable
DEBUG: Filesystem /dev/hugepages: not reflink capable
DEBUG: Filesystem /dev/mqueue: not reflink capable
DEBUG: Filesystem /proc/sys/fs/binfmt_misc: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/blkio (#44); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/blkio: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/hugetlb (#43); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/hugetlb: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/devices (#42); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/devices: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/memory (#41); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/memory: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/freezer (#40); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/freezer: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/net_cls,net_prio (#39); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/net_cls,net_prio: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/perf_event (#38); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/perf_event: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/rdma (#37); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/rdma: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/cpuset (#36); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/cpuset: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/pids (#35); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/pids: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/cpu,cpuacct (#34); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/cpu,cpuacct: not reflink capable
DEBUG: Filesystem /sys/fs/bpf: not reflink capable
DEBUG: Filesystem /sys/fs/pstore: not reflink capable
DEBUG: `cgroup` mount detected at /sys/fs/cgroup/systemd (#31); Ignoring all files in it.
DEBUG: Filesystem /sys/fs/cgroup/systemd: not reflink capable
DEBUG: Filesystem /sys/fs/cgroup/unified: not reflink capable
DEBUG: Filesystem /sys/fs/cgroup: not reflink capable
DEBUG: Filesystem /dev/pts: not reflink capable
DEBUG: Filesystem /dev/shm: not reflink capable
DEBUG: Filesystem /sys/kernel/security: not reflink capable
DEBUG: Filesystem /: reflink capable
DEBUG: Filesystem /sys/firmware/efi/efivars: not reflink capable
DEBUG: Filesystem /run: not reflink capable
DEBUG: `devtmpfs` mount detected at /dev (#5); Ignoring all files in it.
DEBUG: Filesystem /dev: not reflink capable
DEBUG: Filesystem /sys: not reflink capable
DEBUG: `proc` mount detected at /proc (#20); Ignoring all files in it.
DEBUG: Filesystem /proc: not reflink capable
DEBUG: 00:58                           /sys/fs/fuse/connections -> 00:00 fusectl      (underlying disk: unknown; rotational: yes)
DEBUG: 00:57                                /run/user/1000/gvfs -> 00:00 gvfsd-fuse   (underlying disk: unknown; rotational: yes)
DEBUG: 00:56                                     /run/user/1000 -> 00:56 tmpfs        (underlying disk: tmpfs; rotational:  no)
DEBUG: 00:53                            /var/lib/nfs/rpc_pipefs -> 00:00 sunrpc       (underlying disk: unknown; rotational: yes)
DEBUG: 00:50                                          /mnt/data -> 00:00 data         (underlying disk: unknown; rotational: yes)
DEBUG: 259:01                                              /boot -> 259:00 /dev/nvme0n1p1 (underlying disk: nvme0n1; rotational:  no)
DEBUG: 00:48                                      /proc/fs/nfsd -> 00:00 nfsd         (underlying disk: unknown; rotational: yes)
DEBUG: 00:19                                 /sys/kernel/config -> 00:00 configfs     (underlying disk: unknown; rotational: yes)
DEBUG: 00:47                                               /tmp -> 00:47 tmpfs        (underlying disk: tmpfs; rotational:  no)
DEBUG: 00:11                                /sys/kernel/tracing -> 00:00 tracefs      (underlying disk: unknown; rotational: yes)
DEBUG: 00:06                                  /sys/kernel/debug -> 00:00 debugfs      (underlying disk: unknown; rotational: yes)
DEBUG: 00:46                                     /dev/hugepages -> 00:00 hugetlbfs    (underlying disk: unknown; rotational: yes)
DEBUG: 00:18                                        /dev/mqueue -> 00:00 mqueue       (underlying disk: unknown; rotational: yes)
DEBUG: 00:45                           /proc/sys/fs/binfmt_misc -> 00:00 systemd-1    (underlying disk: unknown; rotational: yes)
DEBUG: 00:44                               /sys/fs/cgroup/blkio -> 00:44 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:43                             /sys/fs/cgroup/hugetlb -> 00:43 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:42                             /sys/fs/cgroup/devices -> 00:42 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:41                              /sys/fs/cgroup/memory -> 00:41 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:40                             /sys/fs/cgroup/freezer -> 00:40 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:39                    /sys/fs/cgroup/net_cls,net_prio -> 00:39 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:38                          /sys/fs/cgroup/perf_event -> 00:38 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:37                                /sys/fs/cgroup/rdma -> 00:37 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:36                              /sys/fs/cgroup/cpuset -> 00:36 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:35                                /sys/fs/cgroup/pids -> 00:35 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:34                         /sys/fs/cgroup/cpu,cpuacct -> 00:34 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:33                                        /sys/fs/bpf -> 00:00 none         (underlying disk: unknown; rotational: yes)
DEBUG: 00:32                                     /sys/fs/pstore -> 00:00 pstore       (underlying disk: unknown; rotational: yes)
DEBUG: 00:31                             /sys/fs/cgroup/systemd -> 00:31 cgroup       (underlying disk: cgroup; rotational:  no)
DEBUG: 00:30                             /sys/fs/cgroup/unified -> 00:00 cgroup2      (underlying disk: unknown; rotational: yes)
DEBUG: 00:29                                     /sys/fs/cgroup -> 00:29 tmpfs        (underlying disk: tmpfs; rotational:  no)
DEBUG: 00:28                                           /dev/pts -> 00:00 devpts       (underlying disk: unknown; rotational: yes)
DEBUG: 00:27                                           /dev/shm -> 00:27 tmpfs        (underlying disk: tmpfs; rotational:  no)
DEBUG: 00:07                               /sys/kernel/security -> 00:00 securityfs   (underlying disk: unknown; rotational: yes)
DEBUG: 00:26                                                  / -> 259:00 /dev/nvme0n1p2 (underlying disk: nvme0n1; rotational:  no)
DEBUG: 00:23                          /sys/firmware/efi/efivars -> 00:00 efivarfs     (underlying disk: unknown; rotational: yes)
DEBUG: 00:22                                               /run -> 00:00 run          (underlying disk: unknown; rotational: yes)
DEBUG: 00:05                                               /dev -> 00:05 dev          (underlying disk: dev; rotational:  no)
DEBUG: 00:21                                               /sys -> 00:21 sys          (underlying disk: sys; rotational:  no)
DEBUG: 00:20                                              /proc -> 00:20 proc         (underlying disk: proc; rotational:  no)
DEBUG: Created new RmMDSDevice for rotational disk #0
DEBUG: Starting MDS scheduler with 2 threads
DEBUG: Starting disk 0 (pointer 0x55a5dee6e000) thread #1
DEBUG: Starting disk 0 (pointer 0x55a5dee6e000) thread #2
DEBUG: Freeing device 0 (pointer 0x55a5dee6e000)
DEBUG: Found 273740 files, ignored 1284 hidden files and 328 hidden folders
DEBUG: List build finished at 1.591 with 273740 files
DEBUG: initial size sort finished at time 1.744; sorted 273740 files
free(): invalid pointer
ERROR: Aborting due to a fatal error. (signal received: Aborted)
ERROR: Please file a bug report (See rmlint -h)
> $ rmlint --version
version 2.10.1 compiled: Jun 16 2020 at [09:25:31] "Ludicrous Lemur" (rev unknown)
compiled with: +mounts +nonstripped +fiemap +sha512 +bigfiles +intl +replay +xattr +btrfs-support

rmlint was written by Christopher <sahib> Pahl and Daniel <SeeSpotRun> Thomas.
The code at https://github.com/sahib/rmlint is licensed under the terms of the GPLv3.
> $ uname -a
Linux nas 5.7.4-arch1-1 #1 SMP PREEMPT Thu, 18 Jun 2020 16:01:07 +0000 x86_64 GNU/Linux
sahib commented 4 years ago

Hello @austinbutler,

I need a backtrace to track this down. Please clone rmlint as described here, then...:

sahib commented 4 years ago

Nevermind, I found it. The issue is the csv formatter. It was fixed in f4c821c6d0dc6fdb339a82913f5abbafabe2d4b4. Please double check that it was by fixed by retrying after a recompile.

austinbutler commented 4 years ago

Yep that works. Thanks!