programster / zfs-balancer

php script created in order to try and balance a ZFS RAID array that was built over time by adding drives to the pool over time.
MIT License
43 stars 5 forks source link

File permissions are not maintained #1

Open thehomerepot opened 8 years ago

thehomerepot commented 8 years ago

The script does the following to copy/rename the files. copy($filename, $newname); unlink($filename);

# Rename the file back to what it was originally called. rename($newname, $filename);

You should probably add the following after the copy function to maintain permissions. chown($newname, fileowner($filename)); chgrp($newname, filegroup($filename)); chmod($newname, fileperms($filename));

I'm not a PHP scripter so I just kind of threw this together. There may be an easier way to accomplish this.

jimbobmcgee commented 6 years ago

ACLs, xattrs, modified times and SELinux contexts would also likely be affected here, but PHP has no methods (that I know of) to manipulate them. Possibly overkill but, to that end, chaining some tar calls might be a better option (in particular, GNU tar). Example below might be more driven towards a shell script than a PHP script, but I think the principle is sound...

SUFFIX="!$(basename "$0")-$$-$(date '+%s.%N')"
tar --create --acls --xattrs --selinux --preserve --file=- --add-file="$FILENAME" | tar --extract --acls --xattrs --selinux --preserve --backup --suffix="$SUFFIX"
[ -f "$FILENAME" ] && [ -f "${FILENAME}${SUFFIX}" ] && rm -f -- "${FILENAME}${SUFFIX}"

Now, the multiple calls to tar might be expensive, but you may be able to consolidate them into a single call with judicious use of wildcards or a pre-created --files-from=/tmp/filelist.$$ parameter instead of --add-file=$FILENAME.

That said, not sure how any re-balancing effort would behave with a bunch of snapshots lying around, though (but that's a different problem)...

jimbobmcgee commented 6 years ago

Thinking more about it, from a shell-script perspective, cp may have all we need for this...

SUFFIX="!$(basename "$0")-$$-$(date '+%s.%N')"
cp --preserve=all \
   --no-dereference \
   --force \
   --backup=simple \
   --suffix="$SUFFIX" \
   -- "$FILENAME" "$FILENAME" \
&& rm -f "${FILENAME}${SUFFIX}"

I am working on the basis that, according to the manpage for GNU cp, As a special case, cp makes a backup of SOURCE when the force and backup options are given and SOURCE and DEST are the same name for an existing, regular file. If that holds, then the rename of the existing file, copy to a new file with the same name and removal of the original file can chain together reasonably well.

All that remains is a loop through a list of files, probably obtained via find...

jimbobmcgee commented 6 years ago

It might also be wise not to attempt balance files that have hardlinks, otherwise they will be duplicated (the old version will not be deleted, just the current link to it).

if [ "$(stat --format='%h' "$FILENAME")" eq 1 ]; then
   ...
fi

Alternatively, you would have to relink all hardlinks, using some inode check -- find -samefile, perhaps (although -samefile is non-POSIX).

llewxam-kache commented 5 years ago

Has anyone tested this thoroughly? A shell script makes a lot more sense to me than PHP, but I'm very appreciative that someone went out and did the work!

markusressel commented 4 years ago

I have to solve this rebalancing problem for my personal NAS in the coming days, so I went ahead and created a shell script version of this php script at zfs-inplace-rebalancing. Feel free to use it, give feedback or open PRs.