[Feature Request] Alternative storage management libs

[polarathene]

Is there any plans for supporting other storage management libs like SUSE's new(2 years in development) libstorage-ng. While libstorage-ng is C++, there is also blivet which is a python module by storaged/fedora, though I'm not sure if that has any benefits to offer.

The main idea of libstorage-ng compared to libstorage is to keep all information about storage devices in a graph instead of lists

The graph approach may be useful to calamares in someway too? They have an overview with examples and reasoning of switching to using a graph here.

Phoronix article:

Using libstorage-ng will mean the openSUSE installer can move on to offer better Btrfs multi-device support, BCache SSD/HDD hybrid caching, better encryption features, DM-Cache, Further out there is plans around bettering support for SMR host-managed drives, OpenChannel SSDs (LightNVM), better NVMe support, and other advanced disk features that could not be easily implemented with the old libstorage.

openSUSE blogpost:

..as we did this week with a topic that can have a very noticeable impact in the system performance: partitions alignment.

AFAIK, Calamares is currently using KPMcore. Is Calamares quite tied to that for this part of installation or has it been designed where storage managaement lib is flexible to change/support alternatives?

[abucodonosor]

Calamares depends on KPMcore , no easy way to add whatever other backend.. Also I don't think Calamares will benefit in any way from libstorage-ng since is a SUSE thing and it does implement whatever the Suse installer need/want.

Also Calamares and KPMcore developers work together coordinating releases and features etc.. That won't be possible with any other random lib/backend..

[polarathene]

That's alright, I was just curious if Calamares was designed to be flexible in this area or locked to KPMcore.

As for benefits, I assume it's able to do whatever Calamares currently can via KPMcore? The benefits would be all the extra features libstorage-ng will bring that may be much further off before KPMcore supports such:

Btrfs multi-device support, BCache SSD/HDD hybrid caching, better encryption features, DM-Cache, Further out there is plans around bettering support for SMR host-managed drives, OpenChannel SSDs (LightNVM)

I assume KPMcore doesn't support any of that at present, and may be a while off if at all for some features? Not sure about the partition alignment improvements(they cover that in more detail over the openSUSE blogpost).

Also Calamares and KPMcore developers work together coordinating releases and features etc.. That won't be possible with any other random lib/backend..

I don't see why that couldn't continue to be the case while having third-party support for other backends(if it were possible). SUSE are pretty friendly bunch, I'm sure they'd be receptive to feedback and possibly any use cases Calamares brings up, especially if they see value in it themselves.

---

Closing as this isn't a feature that Calamares is able to support.

[kkofler]

Actually, KPMcore is only currently used in one module, the partition module. So it would be possible to implement a different partition module based on another backend. That said, the partition module is probably the most complex module in all of Calamares, not that easy to rewrite.

In addition, the current plan is actually to rely more on KPMcore. In particular, the requirement checker in the welcome module currently uses libparted (directly), so it was proposed to use KPMcore instead, so that there cannot be spurious differences between what the requirement checker sees and what the actual partition module sees (and also in order to reduce code duplication). Of course, this is something that libstorage-ng could also provide, but that would already be a second module to port.

[polarathene]

So it would be possible to implement a different partition module based on another backend. That said, the partition module is probably the most complex module in all of Calamares, not that easy to rewrite.

Good to know!

Of course, this is something that libstorage-ng could also provide, but that would already be a second module to port.

Yes as the link to blogpost mentions, libstorage-ng already verifies it's partitions, that everything is correct before progressing. Moving the libparted functionality into KPMcore makes perfect sense :)

[adriaandegroot]

Reopening as "interesting feature request". There's two ways to go about this:

• Get Calamares to use par-thing directly. This means writing an alternative to the partition module, which would then live in src/modules .. I'm a fan of keeping things upstream as much as possible, especially now modules can easily make themselves optional based on their dependencies. This would entail writing a ViewStep module, doing a UI to allow selecting things / doing the partitioning (maybe steal code from YAsT, which might have a Qt-based UI already), and defining jobs to run.
• Get KPMCore to use par-thing directly. It supports multiple backends, currently via libparted and sfpart, I think. There's also a dummy backend (which does nothing) and a FreeBSD backend is planned (probably using geom). Then you keep the Calamares code, but need to wrap par-thing in a form that KPMCore understands.

Either would work, it's mostly a matter of figuring out where in the stack you want to live.

[kkofler]

YaST uses the libyui abstraction, not Qt directly, and it's mostly written in Ruby, so I don't think YaST code will be of much use here.

[polarathene]

Just sharing an update of some links for anyone interested in progression of libstorage-ng. Some is perhaps more to do with their installer frontend, I'm not sure how much relates to libstorage-ng, but it might be useful to Calamares devs?:

Changes in the Partitioner UI to Unleash the Storage-ng Power. Showcases some features that they previously weren't able to do with the old partition, involving some software RAID and LVM use-cases and an issue they encountered with supporting BCache and how they resolved that. Another feature was "a disk formatted to host a file-system directly, with no partitions in between". If the BCache insights are useful, they've also blogged some more about that

Intel Rapid Start support.

Supporting Dell hardware for software RAID but firmware driven configuration, plus USB storage devices. The dell support is perhaps niche, and I haven't done any recent installs to target a USB stick(I remember in the past Calamares would forbid certain USB devices such as when they had a live installer), the support allows any device apart from the one running the installer(unless it's a network install, then it permits overwriting itself).

Raspberry Pi installer support. "No custom specific ISO to install from, no precooked image to be copied, just taking the standard unmodified SLE or openSUSE ISO image and installing like you would do in any other computer.". Seems like more of a distro thing than something Calamares would have any involvement in? But if a maintainer is more familiar with what that blogpost discusses, if Calamares would play a part in that for a distro developer to get working, it'd be nice to see such support for SBCs in general.

[demmm]

This feature request is now well over 4 years old, never a PR submitted for it, time to close.