search

lxc / incus

Powerful system container and virtual machine manager
https://linuxcontainers.org/incus
Apache License 2.0
2.47k stars 198 forks source link

Add a `bcachefs` storage driver #213

Open stgraber opened 10 months ago

stgraber commented 10 months ago

Linux 6.7 should finally have initial support for bcachefs so we should be able to add a storage driver for it, roughly based on the btrfs or zfs one (whichever ends up closest).

@RAOF showed some initial implementation of that for LXD a while back, so that shouldn't take too much effort to get that in place and set up some testing for it.

Marking as blocked until Linux 6.7 is released.

stgraber commented 10 months ago

@RAOF did you push your LXD branch anywhere public?

RAOF commented 10 months ago

I did not. I should dust that off and put it somewhere useful!

FlyingWombat commented 8 months ago

bcachefs is now in mainline kernel v6.7 , released 2024-01-07. It is supported by at least Archlinux and Nixos as of the v6.7 kernel release. Bcachefs is feature equivalent with Btrfs and Zfs, as far as LXD is concerned -- with the exception of a dedicated subvolume send/receive mechanism.

CharanSriramUni commented 5 months ago

Hey! Was also interested in approaching this issue.

Is there any additional info available/any good resources to look at for reference?

stgraber commented 5 months ago

That one is likely to be a rather large one to work on.

I've not actually done much research into bcachefs as it's such a new feature (Linux 6.7) but I'd probably start any work on this by first familiarizing myself with the filesystem and its tools, then would probably pick the existing btrfs storage driver (internal/storage/drivers/driver_btrfs*) as a rough template for the work, looking for equivalent commands and logic under bcachefs.

ktran1403 commented 5 months ago

Hello, my group is interested in working on this issue. Is it possible for us to be assigned to this? Thanks.

stgraber commented 5 months ago

@ktran1403 given my previous comment, I suspect it would be very unlikely that your group would have sufficient time to bring this to completion. I estimate this work at a couple of weeks of full time work by someone already familiar with both bcachefs and Incus.

ktran1403 commented 5 months ago

@stgraber I see. Thanks for the heads-up.

milaiwi commented 4 months ago

Since no one else is working on this, would it be possible to break this into separate tasks/issues so we could tackle a somewhat difficult aspect of this project? We've familiarized ourselves with the codebase already given our other tasks but would love to contribute more to this project. If that's possible let us know, if not we understand!

stgraber commented 4 months ago

@milaiwi the first part would be getting a system with a functional bcachefs setup, so that means a 6.7 or 6.8 (probably best) Linux kernel and the bcachefs tools setup.

I'd probably then just play a bit with bcachefs, familiarize yourself with the concepts, start creating subvolumes, snapshots, setting up quotas, ... At which point you should have a good idea of what commands need to be run to deal with it, which will then have to be added to the new driver.

TinkeringWithUS commented 4 months ago

Can you give an example of how we would test out the new driver and whether if Incus is detecting Bcachefs as a valid filesystem?

stgraber commented 4 months ago

Once you've added a minimal driver for it, restarting incus should then have the driver's load function be called and if that succeeds, it will show up in the incus info data.

milaiwi commented 4 months ago

Yeah, we gave a stab at this but this looks like something that would need people who are more comfortable with bcachefs, as you mentioned. From what I've read bcachefs is pretty different from btrfs so we would have had to do a lot more designing.

stgraber commented 4 months ago

Yeah, it shouldn't be too too different from btrfs but it's definitely a fully new storage driver with a different way to create the storage and different commands for most of the other interactions. I also expect things like migration and quotas to work pretty differently.

ktran1403 commented 4 months ago

Hello, while this issue is no longer possible for us to finish in time for class, I still find it rather interesting. Would it be possible for me to continue working on this? So far, I've familiarized myself with bcachefs and managed to get Incus to recognize bcachefs as well as create volumes and snapshots, but there's still a lot of work to be done.

stgraber commented 4 months ago

@ktran1403 for sure! I've assigned it to you so anyone else looking at our open issues is aware that you've started some work on this.

If you end up not having the time to work on this, just drop whatever notes you have in this issue and someone else can take it over.

thomasdba commented 1 week ago

any updates？