search

arkanelinux / arkdep

Toolkit for building, deploying and maintaining an immutable, atomic, btrfs-based system
GNU General Public License v3.0
96 stars 7 forks source link

Difference with openSUSE's immutability tooling #3

Open Malix-Labs opened 1 month ago

Malix-Labs commented 1 month ago

Hello folks !

Is there any documentation about a comparison between arkdep and openSUSE immutability tooling ?

Thanks !

dennis1248 commented 1 month ago

Not yet. But basically, the tech powering both is the same. Both do trickery with Btrfs subvolumes.

MicroOS makes a snapshot of root and then runs a traditional update on the snapshot. We instead work with subvolume exports which are build on the server and then downloaded in full by the user. Updates are a lot bigger, but the image provider is capable of making virtually any change to the image configuration.

Malix-Labs commented 1 month ago

Interesting

Have you discussed with openSUSE to share efforts on those tooling ?

dennis1248 commented 1 month ago

I think the implementation is different enough that there is nothing to share.

user1-github commented 1 month ago

As someone who has previously used Aeon (OpenSUSE's immutable Gnome variant), I'd like to add that in the long run Arkdep is probably more efficient than btrfs snapshots in terms of disk space usage.

While technically each individual btrfs snapshot is more efficient (each new snapshot is just the difference from the previous snapshot), the downside of btrfs snapshot is that they add up over time and after some time they end up using many gigabytes of disk space. AFAIK, at some point old snapshots should supposedly be automatically removed, but to me it hasn't happened once after a few months of using Aeon and I gained over 20 snapshots which ended up using over 10gb of disk space. I've even seen some users of other OpenSUSE variants complaining that they ran out of disk space because of btrfs snapshots. I heard you can manually remove old snapshots, but it's very risky.

Arkdep on the other hand works more similarly to Fedora Silverblue - you only have a fixed amount of deployments (usually just 2 or 3), so most of the time system updates wouldn't add up disk space usage.

dennis1248 commented 1 month ago

Aeon does not clear the old snapshots out automatically?

I actually thought Arkdep would be the less efficient one, we do not do any deduplication, which you do get for free when creating snapshots.

user1-github commented 1 month ago

Aeon supposedly should clear old snapshots automatically at some points, but as I said, it didn't happen during the few months I was using it, so I assume it just happens rarely. So most of the time you end up having dozens of snapshots and 99% chance you wouldn't need most of them.

As a result, even if snapshots are technically more efficient due to deduplication, dozens of them end up taking more disk space than just 2 or 3 Arkdep deployments without deduplication.

I'd say of all immutable systems/toolkits, Fedora Silverblue's rpm-ostree system is probably the most disk space efficient because it has deduplication and it keeps just 2 deployments by default. But rpm-ostree is also the most complex system and it's tied to Fedora. That's why I think Arkdep is the sweet spot of disk space efficiency and simplicity because it keeps only 2 or 3 deployments by default like rpm-ostree and it's more disk space efficient than btrfs snapshots and ABRoot.

Malix-Labs commented 1 month ago

That experience is not the normal behavior and would be valid to take as a neutral comparison to evaluate the storage efficiency between aeon and arkdep

ostree system is probably the most disk space efficient because it has deduplication and it keeps just 2 deployments by default

It is theoretically possible to set to keep an infinite amount of deployment with OSTree, and also set to keep only 2 deployments on Aeon

OSTree […] is more disk space efficient than btrfs snapshots and ABRoot

ABRoot is indeed not storage efficient because it keeps two slightly different Roots (A and B), and I think doesn't employ deduplication (yet, but might be improved)

Because of that, arkdep and ABRoot should be at the same level of magnitude about storage efficiency

user1-github commented 1 month ago

It is theoretically possible to set to keep an infinite amount of deployment with OSTree, and also set to keep only 2 deployments on Aeon

I don't know if it's possible to configure btrfs snapshots so that only the current and 1 or 2 previous snapshots will be kept. At least I tried searching about it and coupdn't find any information.

ABRoot is indeed not storage efficient because it keeps two slightly different Roots (A and B), and I think doesn't employ deduplication (yet, but might be improved). Because of that, arkdep and ABRoot should be at the same level of magnitude about storage efficiency

Last time VanillaOS team has said that they had 2 20gb root partitions, which is way less efficient than Arkdep. I just read that they reduced the size of the root partitions in VanillaOS 2, but it wasn't mentioned by how much.

dennis1248 commented 1 month ago

I don't know if it's possible to configure btrfs snapshots so that only the current and 1 or 2 previous snapshots will be kept. At least I tried searching about it and coupdn't find any information.

That is something the wrappers scripts around it will handle.

Last time VanillaOS team has said that they had 2 20gb root partitions, which is way less efficient than Arkdep. I just read that they reduced the size of the root partitions in VanillaOS 2, but it wasn't mentioned by how much.

The new Vanilla OS works slightly different, it instead uses LVM for its volumes. And LVM probably has tricks to minimize this.