Feature: Possibility for rename resources, snapshots and storage pools

[kvaps]

According #17 I've used two different storage pools and call auto-placing command two times, for place data on two different racks.

But few time ago there was released linstor CSI plugin which allows to set AUX properties: ReplicasOnSame and ReplicasOnDifferent, what made my setup not so optimal as just using AUX properties.

I need to rename my current storage pools for allow using same pool name but with differnet aux properties.

[ghernadi]

Currently Linstor does not support any kind of renaming action.

If you really have to do this, you have two options:

• NOT RECOMMENDED: you do it manually by editing the database (NOT recommended, as you have to be very careful to update all references to the changed name)
• a bit "more" recommended: delete your current database (or perform "lostNode" commands for the affected nodes) and recreate using migration everything with the new storage pool names.

Note: we do have internal discussions if and how we could provide renaming actions. That means this situation might be easier to resolve in (far) future.

[kvaps]

Just mention that resource-groups also needs this feature

[raltnoeder]

Renaming resources is virtually impossible, first due to internal constraints in LINSTOR (it would have to keep both resource names until the resource was successfully renamed everywhere, but the data structures and the database constraints do not allow holding incomplete or invalid values), and what's even more important is that LINSTOR is managing external resources, like LVM, ZFS, DRBD, etc., and neither the operating system nor DRBD or any of the other components around it are properly designed to support on-the-fly renaming of devices.

[volkerjaenisch]

@raltnoeder So this looks like a piece of not well designed software to me. Renaming of resources is not a brand new feature, but an really old issue. In DB terms no one would use the name of a resource to reference it. You would use a kind of primary key to share the resource it with other entities. So that you can easily rename the resource. In my case I do not use LVM and ZFS. So in the end it comes down to linstore and drbd which are both from the same company. And at least these two bits of software could be improved for renaming resources. Cheers, Volker

[raltnoeder]

I designed the software (LINSTOR), and the main priority when the project was started in 2016 was better reliability and maintainability, because the typical use case for DRBD is for higher availability. Availability is a function of the time it takes to resolve an outage, or to perform maintenance on a system. Complexity is the natural enemy of administrators as well as software developers, and that includes data that is full of surrogate keys consisting of meaningless numbers and letters. DRBD did not have the ability to rename resources back then, and this functionality was only added very recently, and even now is not without corner-cases and limitations (and also limited to the local node, it can't rename across the entire cluster in a transaction). For example, resources are associated with various paths in the Linux filesystem, e.g. in /dev, but also in /sys, and those paths can not be changed while any process is accessing them. If you think that's not well-designed, then yes, maybe it isn't, but the reason for that is not the way data is stored in the database (that would just be a simple update statement either way), but the fact that the entire environment around LINSTOR isn't well-designed. We are trying to build a cluster-wide transaction-safe storage automation system on top of an operating system and user environment that does not even provide node-local transaction-safety and is full of corner-cases, potential failure modes, undocumented failure modes, race conditions and similar problems. Linstor contains tens of thousands of lines of code just to deal with reacting to and correcting those problems. And this is the main reason why we can't just add a feature that makes a cluster-wide change in a day and 50 lines of code.

[volkerjaenisch]

Dear Robert!

On 20.05.21 17:12, Robert Altnoeder wrote:

I designed the software (LINSTOR), and the main priority when the project was started in 2016 was better reliability and maintainability, because the typical use case for DRBD is for higher availability. Availability is a function of the time it takes to resolve an outage, or to perform maintenance on a system. Complexity is the natural enemy of administrators as well as software developers, and that includes data that is full of surrogate keys consisting of meaningless numbers and letters. DRBD did not have the ability to rename resources back then, and this functionality was only added very recently, and even now is not without corner-cases and limitations (and also limited to the local node, it can't rename across the entire cluster in a transaction). For example, resources are associated with various paths in the Linux filesystem, e.g. in /dev, but also in /sys, and those paths can not be changed while any process is accessing it. If you think that's not well-designed, then yes, maybe it isn't, but the reason for that is not the way data is stored in the database (that would just be a simple update statement either way), but the fact that the entire environment around LINSTOR isn't well-designed. We are trying to build a cluster-wide transaction-safe storage automation system on top of an operating system and user environment that does not even provide node-local transaction-safety and is full of corner-cases, potential failure modes, undocumented failure modes, race conditions and similar problems. Linstor contains tens of thousands of lines of code just to deal with reacting to and correcting those problems. And this is the main reason why we can't just add a feature that makes a cluster-wide change in a day and 50 lines of code.

I am sure you did your best in a really hostile environment. Thank you for your work.

I wonder why you did not include an abstraction? If drbd was not able to distinguish between a "resource label" and a "resource identifier" then linstore would be exactly the place where to abstract from DRBD (and the other resources) identifiers, to get more degrees of freedom to work with. Imagine DRBD would have worked with GUIDs as "resource identifiers". Nobody likes to remember GUIDs so one makes an abstraction and map a user friendly labels to the GUIDs and you have done the world a big favour. Think of IPs and FQHNs.

But you choose to hard wire the "resource label" to the DRBD "resource identifier" and therefore enforced a strong coupling which now is part of the problem. You are absolutely right that this is not the only problem.

Maybe you do not understand the need we customers have. We do not want to rename a DRBD "resource identifier". We simply want an abstraction for the user space tools where we work with "resource labels" which are then mapped to the true "resource identifiers".

I still not understand why it should not be possible to add this abstraction. You do not have to even talk to the backend (DRBD, linux kernel etc.) to come up with an abstraction which maps a user friendly and easy to rename "resource label" which are then mapped (bidirectional) to the true "resource identifier". The "resource label" and its mapping has to be included in the user space tools.

Surely even this solution is surely not the work of days and 50 lines of code. I am developer for 40 years, and know what such code changes imply: API Changes, DB changes, protocol changes. I have done more than one project where the primary entity changed (due to unforeseeable reasons) so that a complete redesign of code, API, DB, Protocols was necessary - sometimes programming is truly hard. But it is never "virtually impossible" to change software. The changes may have a huge price tag but they are doable.

There are several users out there which have the same problem with renaming "resource labels" (please visit the recent discussion in the drbd-users list) and are looking for a solution for this particular corner case.

Cheers,

Volker

--

========================================================= inqbus Scientific Computing Dr. Volker Jaenisch Hungerbichlweg 3 +49 (8860) 9222 7 92 86977 Burggen https://inqbus.de

[raltnoeder]

The strong coupling between resource names in LINSTOR, DRBD and the backing volumes exists for maintainability. Most administrators are already overwhelmed with using the system as it is, and it seems that most people don't read the manual. Now imagine an administrator trying to fix 900 resources on a system where he or she has to deal with at least 3 different names for the different layers of the storage system - LINSTOR, DRBD, backing storage - trying to figure out which DRBD resource belongs to which LINSTOR resource and why it doesn't attach to the backing storage that has yet another name. Whenever that happens, it's simply a downtime multiplier. There also did not seem to be a reason to rename resources as a part of daily business - after all, DRBD did not have the ability to rename resources at runtime for more than 15 years, and apparently, noone was missing that feature. In the few cases where a major change to infrastructure would suggest renaming a resource to reflect the change, a manual intervention would have seemed sufficient. If lots of resources need to be renamed all the time as a part of daily business, that's quite certainly the result of bad planning.