search

sahib / rmlint

Extremely fast tool to remove duplicates and other lint from your filesystem
http://rmlint.rtfd.org
GNU General Public License v3.0
1.91k stars 132 forks source link

False empty directories #459

Closed rhinot closed 3 years ago

rhinot commented 3 years ago

Description

Basic rmlint search on a directory results in a report of an empty directory, despite directory not being empty.

Output seen

$ rmlint /mnt/pool/dropbox/

# Empty dir(s):
    rmdir '/mnt/pool/dropbox'

==> 1 file(s) after investigation, nothing to search

Looking at the directory (a mergerfs pool of USB drives):

$ ls /mnt/pool/dropbox/
1970  2017         rpd-tmp-0T04gu  rpd-tmp-AHs9ub  rpd-tmp-fco23c  rpd-tmp-HSY13n  rpd-tmp-lkyhBg  rpd-tmp-r7JTZr  thumbnails-digikam.db
2013  2018         rpd-tmp-4OkyR9  rpd-tmp-aTRTRn  rpd-tmp-FSjn0k  rpd-tmp-ht1zzl  rpd-tmp-mCJ_vH  rpd-tmp-RVGuk7
2014  2019         rpd-tmp-7H60iU  rpd-tmp-cDYiEr  rpd-tmp-GouWFZ  rpd-tmp-iczyYs  rpd-tmp-oecn6_  rpd-tmp-sdmNEf
2015  aaa.mp4      rpd-tmp-8icawh  rpd-tmp-dt0WaY  rpd-tmp-H2qynB  rpd-tmp-kaPvvu  rpd-tmp-PeWk1z  rpd-tmp-T0NoQo
2016  digikam4.db  rpd-tmp-ABSKqu  rpd-tmp-EGjCmC  rpd-tmp-HKGiK1  rpd-tmp-lF7Ce9  rpd-tmp-pP8YdT  rpd-tmp-tmbJ3X

Expected output

rmlint to run over directory

Alternatives tried

$ rmlint /mnt/pool/dropbox/*

# Empty dir(s):
    rmdir '/mnt/pool/dropbox/rpd-tmp-tmbJ3X'
    rmdir '/mnt/pool/dropbox/rpd-tmp-sdmNEf'
    rmdir '/mnt/pool/dropbox/rpd-tmp-r7JTZr'
    rmdir '/mnt/pool/dropbox/rpd-tmp-pP8YdT'
    rmdir '/mnt/pool/dropbox/rpd-tmp-oecn6_'
    rmdir '/mnt/pool/dropbox/rpd-tmp-mCJ_vH'
    rmdir '/mnt/pool/dropbox/rpd-tmp-lkyhBg'
    rmdir '/mnt/pool/dropbox/rpd-tmp-lF7Ce9'
    rmdir '/mnt/pool/dropbox/rpd-tmp-kaPvvu'
    rmdir '/mnt/pool/dropbox/rpd-tmp-iczyYs'
    rmdir '/mnt/pool/dropbox/rpd-tmp-ht1zzl'
    rmdir '/mnt/pool/dropbox/rpd-tmp-fco23c'
    rmdir '/mnt/pool/dropbox/rpd-tmp-dt0WaY'
    rmdir '/mnt/pool/dropbox/rpd-tmp-cDYiEr'
    rmdir '/mnt/pool/dropbox/rpd-tmp-aTRTRn'
    rmdir '/mnt/pool/dropbox/rpd-tmp-T0NoQo'
    rmdir '/mnt/pool/dropbox/rpd-tmp-RVGuk7'
    rmdir '/mnt/pool/dropbox/rpd-tmp-PeWk1z'
    rmdir '/mnt/pool/dropbox/rpd-tmp-HSY13n'
    rmdir '/mnt/pool/dropbox/rpd-tmp-HKGiK1'
    rmdir '/mnt/pool/dropbox/rpd-tmp-H2qynB'
    rmdir '/mnt/pool/dropbox/rpd-tmp-GouWFZ'
    rmdir '/mnt/pool/dropbox/rpd-tmp-FSjn0k'
    rmdir '/mnt/pool/dropbox/rpd-tmp-EGjCmC'
    rmdir '/mnt/pool/dropbox/rpd-tmp-AHs9ub'
    rmdir '/mnt/pool/dropbox/rpd-tmp-ABSKqu'
    rmdir '/mnt/pool/dropbox/rpd-tmp-8icawh'
    rmdir '/mnt/pool/dropbox/rpd-tmp-7H60iU'
    rmdir '/mnt/pool/dropbox/rpd-tmp-4OkyR9'
    rmdir '/mnt/pool/dropbox/rpd-tmp-0T04gu'
    rmdir '/mnt/pool/dropbox/2019'
    rmdir '/mnt/pool/dropbox/2018'
    rmdir '/mnt/pool/dropbox/2017'
    rmdir '/mnt/pool/dropbox/2016'
    rmdir '/mnt/pool/dropbox/2015'
    rmdir '/mnt/pool/dropbox/2014'
    rmdir '/mnt/pool/dropbox/2013'
    rmdir '/mnt/pool/dropbox/1970'

==> Note: Please use the saved script below for removal, not the above output.
==> In total 41 files, whereof 0 are duplicates in 0 groups.
==> This equals 0 B of duplicates which could be removed.
==> 38 other suspicious item(s) found, which may vary in size.
==> Scanning took in total 0.193s.

Wrote a sh file to: /home/pi/repos/rmlint/rmlint.sh
Wrote a json file to: /home/pi/repos/rmlint/rmlint.json

Other details

Raspberry Pi 4 Debian Buster

$ rmlint --version
version 2.10.1 compiled: Dec 29 2020 at [16:35:26] "Ludicrous Lemur" (rev 6abfba9a)
compiled with: +mounts -nonstripped +fiemap +sha512 +bigfiles +intl -replay +xattr +btrfs-support
SeeSpotRun commented 3 years ago

Hi, sorry for slow response.

As far as I can see, your dir is full of other dirs, which themselves are empty. In rmlint this is still interpreted as an empty dir since there are no files anywhere. If your dirs are not empty, please reply here and I'll look further.

rhinot commented 3 years ago

Hey Daniel - No worries on timing.

All of these directories are not empty. Some have small images, some (the year based directories) have layers of date-based sub-folders, which have images in them.

SeeSpotRun commented 3 years ago

Ok that's odd.

If you run rmlint -vvv /mnt/pool/dropbox/rpd-tmp-tmbJ3X, do you see any warnings or debug messages about files or folders that are being ignored?

SeeSpotRun commented 3 years ago

@rhinot Can you compile from the develop branch and re-test?

SeeSpotRun commented 3 years ago

unable to reproduce; please re-open issue if required

cebtenzzre commented 2 years ago

Some information that would be useful, if you can still reproduce this: