Path preserving

[devbrian]

Would it be possible to get a path preserving argument when we have multiple RW drives?

For further clarification by what I mean:

Lets say I have this mount setup

RW - /mnt/folderA/ RW - /mnt/folderB/ Both overlayed at - /mnt/merged/

On this command -

mv /mnt/merged/subfolderA/fileA.csv /mnt/merged/subfolderB/fileA.csv

If /mnt/merged/subfolderA/fileA.csv is actually in /mnt/folderA/ then fileA.csv should be moved to /mnt/folderA/subfolderB/fileA.csv

If /mnt/merged/subfolderA/fileA.csv is actually in /mnt/folderB/ then fileA.csv should be moved to /mnt/folderB/subfolderB/fileA.csv

[briankendall]

Not sure if this sort of feature is outside the purview of unionfs, but I too would love to have the option for preserving paths like this.

[rpodgorny]

i have no problem accepting this. i think it is well within the scope of unionfs. the only "problem" i see is maybe a vague problem statement. let's elaborate a bit and try to find some corner cases...!

[briankendall]

Good idea! From the looks of things this feature would only affect rename as file creation or copying would continue working as before. I think what would change is, after unionfs_rename determines what branch the source / from path of a rename operation is in, it would check to see if that same branch a) is RW and b) has a folder tree that includes the path of the destination's folder. If so, it would move the file to that location in the same branch rather than to the first RW branch.

So for an example, if we have a file structure like so:

/folderA/subdirA/subdirC/ /folderB/subdirA/subdirB/file.txt /folderB/subdirA/subdirC/ /union

...then both are mounted to a union filesystem like so:

unionfs /folderA=RW:/folderB=rw /union

...and finally file.txt is moved like so:

mv /union/subdirA/subdirB/file.txt /union/subdirA/subdirC/file.txt

Then unionfs_rename would find that /union/subdirA/subdirB/file.txt is contained within branch 2, and since branch2 is RW and contains subdirA/subdirC, it would move the file to /folderB/subdirA/subdirC/file.txt rather than to /folderA/subdirA/subdirC/file.txt.

Similarly, if the move operation's destination is a directory, i.e. mv /union/subdirA/subdirB/file.txt /union/subdirA/subdirC/ the same thing would occur.

Additionally, I'm thinking this feature would create the necessary folders if the path in the source branch doesn't exist, or at least have the option to do so, so that it keeps things on the same disk as much as possible. This could be very useful for a union filesystem that merges together different physical drives that contain very large files, since otherwise a single move operation could then involve copying one of them from one drive to another rather than being instantaneous. To illustrate, consider the following file structure:

/folderA/subdirA/subdirC/ /folderB/subdirA/subdirB/file.txt /union

It's mounted into a union filesystem that merges folderA and folderB like before, and then file.txt is moved like so:

mv /union/subdirA/subdirB/file.txt /union/subdirA/subdirC/file.txt

Since this time /folderB/subdirA/subdirC/ doesn't exist, unionfs would create /folderB/subdirA/subdirC and then move file.txt into it.

The big edge cases I see here are about what to do when the permissions differ between the different branches. To go back to the first example, what if the permissions were something like this:

r-x------ /folderA/subdirA/subdirC/ rwx------ /folderB/subdirA/subdirB/file.txt rwx------ /folderB/subdirA/subdirC/ rwx------ /union

...Should mv /union/subdirA/subdirB/file.txt /union/subdirA/subdirC succeed? I think in this case, the answer is yes, but I'm curious what you think.

Another edge case, what if the permissions were this:

rwx------ /folderA/subdirA/subdirC/ rwx------ /folderB/subdirA/subdirB/file.txt r-x------ /folderB/subdirA/subdirC/ rwx------ /union

Now moving file.txt into /folderB/subdirA/subdirC/ would fail. So should mv /union/subdirA/subdirB/file.txt /union/subdirA/subdirC fail too? Or should unionfs move file.txt into /folderA/subdirA/subdirC/ instead? This one I'm not so sure about.

[briankendall]

It's over two years later but suddenly I found myself in need of unionfs again, and also found that it didn't behave quite the way I needed it to because of not having a path preserving feature. Particularly in macOS there's things that don't work properly without it. For example, when mounting a union filesystem as a local drive, the system wants to create hidden folders for files that are moved to the trash can, and that doesn't work properly without a path preserving feature that creates the needed folders on the file's current branch.

So I went ahead and hacked in path preserving. You can see my work here: https://github.com/briankendall/unionfs-fuse/commits/bribri

Specifically, these commits add a new option preserve_branch. When used, any time a file in a RW branch is moved, it'll stay in the same branch. Folders are created as needed to accomplish this.

That git branch also has a few macOS specific fixes for building and the tests. I pulled my changes into a Debian 11.3 system and ran the tests there, and they all passed, so hopefully this works as intended in Linux too.

@rpodgorny Would you like me to make a PR for this work? I'd clean up the git branch a bit first, and of course you can request changes if you like.

[rpodgorny]

hi @briankendall ! ...thanks for your effort! i've created a "bribri" branch with your work a some petty whitespace patches of mine... ...i'll be leaving for vacation in two days so i have some catching up to do at work but i plan to merge this after i return...

meanwhile, i have these questions: 1) what is "-o nobrowse" mount parameter? 2) is your cmake patch needed for you to compile on your system? i've implemented some changes changes for macos in the past and i thought everything works as-is... 3) do you also have some anti-tests? something that is expected to fail when "preserve_branch" is not used?

[rpodgorny]

hmmm, i don't see @aakefbs commenting here... ...so just mentioning. ;-)

[bsbernd]

I think branch preserving is a good idea. I will look at the patches during the next days.

[briankendall]

1. -o nobrowse is a mac-only fuse option (I think). It mainly seems to stop icons appearing on the desktop for every single union filesystem that gets mounted.
2. Yes, on my system it couldn't find libfuse without those changes.
3. No, I didn't write any anti-tests! I'll add some of those in the near future.

[briankendall]

Added some anti-tests

[rpodgorny]

finally merged and released in version 3.0 - sorry for the wait and thanks again for your contribution!