akiradeveloper / dm-writeboost

Log-structured Caching for Linux
GNU General Public License v2.0
120 stars 17 forks source link

Transparent migration from linear device to writeboost device #170

Open akiradeveloper opened 7 years ago

akiradeveloper commented 7 years ago

Consider a linear device with linear { offset = n, backing = d } We can transparently migrate the linear device (may be created by lvm) to writeboost device by rewriting the table as writeboost { offset = n, backing = d, cache = c, ... }

AFAIK, transparent migration is provided by enhanceIO (right?)

This is little bit tricky but may help gain more users.

The problem is that writeboost needs another constructor parameter "offset" but it's maybe possible to add this parameter while keeping backward compatibility. (If everyone use dm-writeboost-tools, we don't need to care about this because we can take every controls over low-level dmsetup APIs)

If this feature is favored, I will make a command like "wbmigrate" that takes name of the linear device, path to the cache device and writeboost optionals, and then turns it into writeboost device.

akiradeveloper commented 7 years ago

it's maybe possible to add this parameter while keeping backward compatibility.

We can add "offset n" to static optional parameter

corradofiore commented 7 years ago

You're right that EnhanceIO provides this feature and I agree that it is relevant. As far as I'm concerned, +1.

tanantharaman commented 3 years ago

I was just alerted by onlyjob that his writeboost wrapper script (listed in the dm-writeboost main page as a related project) already allows a backing device with data to be converted to a dm-writeboost device without losing any data.

akiradeveloper commented 3 years ago

@tanantharaman What onlyjob's script is doing is making a new device using two already existing devices but my intention here is rewriting backing device's dm table if it is dm linear device. I know this only works in very specific situation and not in general.

This features helps in such situation: A file system is mounted on a linear device X and the admin want to change it to writeboost device caching on device Y. If we have to make another writeboost device Z from X and Y, this isn't possible without unmounting the filesystem and then remounting it. This operation isn't always allowed but if we could overwrite the X's dm table to make it writeboost device, we can do this without unmounting.

tanantharaman commented 3 years ago

Thanks for the explanation. I can see the advantage of not having to unmount the device X for a local drive, which might require terminating all local jobs with open file handles.

In my case the device X was on the main NFS file server and I was able to unexport X, unmount X, then remount as Z and export Z, without any of the active NFS IO transactions from client machines being affected, except for a brief pause. Even during the time the NFS device X was unmounted, NFS writes were still proceeding and buffering up on the client NFS cache or server side buffer cache. Only reads were hung until the new device Z was mounted and the NFS export activated, but then resumed without any client job IO transaction failure. In a sense NFS provides a wrapper (with data buffering) around the DM-device and ext4 filesystem. This only works if there are no local jobs on the NFS file server accessing the device X.