cross-file system move

[andreasstieger]

mv on the same filesystem is rename(2). mv across file systems is cp followed by rm, a non-atomic operation. Implement mvz with accelerating of the copy and remove phases using mostly the existing code base.

[SUPERCILEX]

Some thoughts.

Implementing this at maximum performance requires a single-pass algorithm (i.e. do the copy + remove in one go for each directory), but that has significant safety implications I'm not comfortable with. What happens when a copy/remove fails? Now you're left with subsets of each directory and trying to rebuild the original directory will be a pain. Also if you want to be 100% correct, you need an fsync between the copy and remove for each file to prevent situations where removal succeeds and copy fails. I wouldn't be comfortable endorsing the max perf algorithm (as much as it bums me out) given its potential for data loss, so that basically means you need a two pass algorithm: copy everything, sync, then remove the old dir. But this can be implemented easily with the shell: cpz ...; sync; rmz ....

So then the question is how often do people perform cross-fs moves? And are they aware of doing so? I could see value in providing mvz if people are unaware that they're doing cross-fs moves and will use mvz by default so it can accelerate those cases. I don't know if that's the case though. Then again, implementing this should be easy so maybe it's worth doing anyway?

On the semantics side:

• rmz requires --force to remove something that doesn't exist.
• cpz requires --force to overwrite a file or attempt to merge a directory.
• mvz should require --force to delete the destination file/dir if it exists.

[andreasstieger]

I think it depends on whether you see the project as a drop-in replacement. I understand that you do not. So I agree that there is not a really compelling use case for an accelerated move across fs that could not be implemented at a similar stability level in the shell.

[andreasstieger]

Just wanted to feed back the idea for consideration. Closing as the suggestion is fully understood and considered, thanks.

[SUPERCILEX]

I think it depends on whether you see the project as a drop-in replacement. I understand that you do not.

Thanks for putting this into words! Hadn't realized that's the philosophy I was using, but you're absolutely right.

I'm open to reconsideration if other people are interested/have use cases.