Permission denied when run in a folder whose parent dir is owned by another group/user

andyslucky commented 1 year ago

Context

I am trying to find duplicates in a subpath, that is a child of a root owned directory. The file structure resembles /mnt/md0/plex/data/Music, and I have permission to cd to any directory in that path. The suffix plex/data/Music is owned by my user, and it has permissions equivalent to chmod 777 -R; however /mnt/md0 is ownded by the root (for obvious reasons) and they both have read permissions set for every user.

Problem

After I cd to the music folder and run fclones group . I am given a permissions warning /mnt/md0/plex/data/Music Permission denied (os error 13), and no files are scanned.

Temporary Solution

Creating a bind mount in my home directory with the following command sudo mount --bind /mnt/md0/plex/data/Music ~/plexMusic resolves the issue.

It seems maybe each component of the path gets walked, rather than starting the walk from the start directory down the tree. https://github.com/pkolaczk/fclones/blob/0345c8ce15735495c0b6bab5f995a3127847624c/src/walk.rs#L219 This would explain why creating a bind mount in my home directory would change this behavior, since this would change the group ownership of the parent node(s) (i.e. /home/myuser instead of /mnt/md0). It would seem the more useful and commonly expected behavior would be "start walking the children of Music directly" e.g. std::fs::read_dir("/mnt/md0/plex/data/Music")

pkolaczk commented 1 year ago

It definitely does not walk starting from root, but I'll check if maybe symbolic link resolution isn't messing things up...

andyslucky commented 1 year ago

Building source and running the executable, I did not encounter the same issue. I installed the binary with snap. Snap version 2.56.2 and fclones version 0.27.1. Not sure what might be the culprit. For now I will build the source and use that :)

pkolaczk commented 1 year ago

snap might be the problem, because it confines the process in a sandbox. Try installing with --classic flag to enable unrestricted access.

pkolaczk commented 1 year ago

I was able to reproduce the issue.

$ fclones group /media/pkolaczk/SanDisk\ 128/
[2022-09-03 12:09:33.473] fclones:  info: Started grouping
[2022-09-03 12:09:33.495] fclones:  warn: Failed to read dir '/media/pkolaczk/SanDisk 128': Permission denied (os error 13)
[2022-09-03 12:09:33.495] fclones:  info: Scanned 1 file entries
[2022-09-03 12:09:33.495] fclones:  info: Found 0 (0 B) files matching selection criteria
...

Installing with --classic doesn't work, but I've found a simpler workaround that works:

$ snap connect fclones:removable-media

Now it is allowed to access removable media - in my case and USB drive:

$ fclones group /media/pkolaczk/SanDisk\ 128/
[2022-09-03 12:13:49.294] fclones:  info: Started grouping
[2022-09-03 12:13:49.296] fclones:  info: Scanned 7 file entries
[2022-09-03 12:13:49.338] fclones:  info: Found 5 (1.4 KB) files matching selection criteria

Now reading the snap docs to see if maybe I can somehow make snap automatically connect to that capability...

pkolaczk commented 1 year ago

I added documentation on how to manually connect removable-media capability and also requested to autoconnect at snapcraft forums here: https://forum.snapcraft.io/t/request-autoconnect-removable-media-for-fclones/31592. However, autoconnecting removable-media requires review of the app so that is not guaranteed and may take some time.

pkolaczk commented 1 year ago

Follow up issue: #154

andyslucky commented 1 year ago

Thanks for looking into this!

remd commented 6 months ago

I wasn't able to use fclones installed with snap on Ubuntu 22.04.4 LTS against a mounted ext4 disk. Even after manually granting snap connect fclones:removable-media I still received a Permission denied (os error 13) error:

root@host:/home/emd# snap connect fclones:removable-media
root@host:/home/emd# fclones link <fclones-dupes.txt
[2024-01-27 00:06:36.856] fclones:  info: Started deduplicating
[2024-01-27 00:06:36.857] fclones: warn: Failed to open file ... for write: Permission denied (os error 13)

I removed the snap-installed fclones and installed fclones-0.34.0-linux-glibc-x86_64.tar and was able to complete the operation:

emd@host:~$ sudo fclones link <fclones-dupes.txt
[2024-01-27 00:13:01.678] fclones:  info: Started deduplicating
[2024-01-27 00:13:33.084] fclones:  info: Processed 927 files and reclaimed 548.9 GB space

The files being deduplicated are on a /mnt representing an HDD owned by me (emd:emd) however the permissions within the directory being deduplicated are a mixture of emd:emd and debian-transmission:debian-transmission.

Everything worked flawlessly with the pre-compiled binary and it really made me appreciate the two-stage approach to the deduplication. Awesome software and many thanks! ❤

pkolaczk / fclones