void-linux / void-mklive

The Void Linux live image maker
https://voidlinux.org
Other
326 stars 190 forks source link

Any way to override automatic LiveOS_rootfs size allocation? #202

Closed Zapeth closed 2 years ago

Zapeth commented 3 years ago

Whenever I run the live image in VirtualBox it seems the assigned system memory of the VM is split about 50:50 in LiveOS_rootfs and tmpfs space, is there any way to override this behavior?

For example I assigned 6GB to the VM, but actual storage in the live session on / is only 3GB, the rest is assigned to tmpfs mounts. I know I dont need more than 1GB of tmpfs space, but would require about 4.5GB "persistent" space.

mobinmob commented 2 years ago

Yes. Please check the rd.live.overlay.* kernel cmdline options that dracut accepts. The current implementation uses the rd.live.overlay.overlayfs=1, but you can use rd.live.overlay.size:

rd.live.overlay.size=<size_MiB>
           Specifies a non-persistent Device-mapper overlay size in MiB.
           The default is 32768.

The rationale for the use of rd.live.overlay.overlayfs=1 is explaned in the original PR that introduced it.

Zapeth commented 2 years ago

The current implementation uses the rd.live.overlay.overlayfs=1, but you can use rd.live.overlay.size:

Do you mean these options are mutually exclusive? I tried appending e.g. rd.live.overlay.size=5000 to the kernel options during boot, but the allocated file system sizes remained unchanged (half of system memory)

I'm also a bit confused by the default value apparently being 32768, shouldn't that already exceed the limit of my use case? (LiveOS_rootfs size 4.5GB and the rest of the available system memory to tmpfs (1.5GB))

mobinmob commented 2 years ago

The current implementation uses the rd.live.overlay.overlayfs=1, but you can use rd.live.overlay.size:

Do you mean these options are mutually exclusive? I tried appending e.g. rd.live.overlay.size=5000 to the kernel options during boot, but the allocated file system sizes remained unchanged (half of system memory)

The two options use different subsystems/strategies for the temporary union filesystem. rd.live.overlay.overlayfs uses overlayfs, rd.live.overlay.size uses a device-mapper snapshot - the default. Please see the relevant section of the dracut.cmdline(7) manpage.

I'm also a bit confused by the default value apparently being 32768, shouldn't that already exceed the limit of my use case? (LiveOS_rootfs size 4.5GB and the rest of the available system memory to tmpfs (1.5GB))

I am not sure what you mean, but the tmpfs size is an upper limit, not an exclusive allocation. I think the same applies to the default overlayfs writable overlay.

Zapeth commented 2 years ago

I am not sure what you mean, but the tmpfs size is an upper limit, not an exclusive allocation. I think the same applies to the default overlayfs writable overlay.

I was just questioning whether the size option would be actually required, if the default value was larger than my use case already.

The two options use different subsystems/strategies for the temporary union filesystem. rd.live.overlay.overlayfs uses overlayfs, rd.live.overlay.size uses a device-mapper snapshot - the default. Please see the relevant section of the dracut.cmdline(7) manpage.

Ok, so I have to disable rd.live.overlay.overlayfs if I want to force a different size? I'm not actually sure if I want that, ideally it should work exactly like it does by default (with the overlayfs), except with a different size allocation ratio. Considering there only seems the be the rd.live.overlay.size option available, I suppose its not possible to change this behavior?

I read the linked section, but it didn't really help me in understanding the issue better (I'm not all that familiar with this kind of stuff in linux). If you have an idea of how a configuration/setup could look like for my use case, could you perhaps list the steps involved to apply it?

mobinmob commented 2 years ago

I am not sure what you mean, but the tmpfs size is an upper limit, not an exclusive allocation. I think the same applies to the default overlayfs writable overlay.

I was just questioning whether the size option would be actually required, if the default value was larger than my use case already.

The curent default behaviour is an easy way to have a sane size of rw space that will cover most uses cases of the live cd. If you require more rw space, you can change it in the grub kernel cmdline. You can do that by replacing rd.live.overlay.overlayfs=1 with rd.live.overlay.size=.

Zapeth commented 2 years ago

The curent default behaviour is an easy way to have a sane size of rw space that will cover most uses cases of the live cd. If you require more rw space, you can change it in the grub kernel cmdline. You can do that by replacing rd.live.overlay.overlayfs=1 with rd.live.overlay.size=.

Maybe I'm completely wrong, but considering what we have discussed so far, I don't think this option helps with my issue:

Looking instead at the explanation in your PR, it looks like this part might be of more interest for me:

The first patch enables the overlayfs support in dracut, which creates and uses a tmpfs with the same size as the one mounted on /run (50% of available memory).

Do you know where the 50% of available memory part originates from and if its configurable/exposable?

mobinmob commented 2 years ago

The curent default behaviour is an easy way to have a sane size of rw space that will cover most uses cases of the live cd. If you require more rw space, you can change it in the grub kernel cmdline. You can do that by replacing rd.live.overlay.overlayfs=1 with rd.live.overlay.size=.

Maybe I'm completely wrong, but considering what we have discussed so far, I don't think this option helps with my issue:

* it implies deactivation of overlayfs

It changes the way dracut provides a writable overlay, from using overlayfs to using dm-snapshot (the default mechanism).

* its default value already exceeds my desired rw space that I want to allocate

I am not sure why that created problems for your usecase, but ok...

* setting it anyway to different lower values has no visible effect in the mounted live image

Looking instead at the explanation in your PR, it looks like this part might be of more interest for me:

The first patch enables the overlayfs support in dracut, which creates and uses a tmpfs with the same size as the one mounted on /run (50% of available memory).

Do you know where the 50% of available memory part originates from and if its configurable/exposable?

Yes, 50% of available memory is the default size for tmpfs.I am not sure it can be configured in this case - that is using overlayfs with dracut.

leahneukirchen commented 2 years ago

tmpfs can be resized with mount -o remount,size=8G /tmp.

Zapeth commented 2 years ago

It changes the way dracut provides a writable overlay, from using overlayfs to using dm-snapshot (the default mechanism).

Yes, and thats why its not useful for me, as I want to keep the overlayfs mechanism. Mostly because I have no experience whatsoever with dm-snapshots, but also because overlayfs already works fine for me otherwise, so being able to use it would be ideal

* its default value already exceeds my desired rw space that I want to allocate

I am not sure why that created problems for your usecase, but ok...

I didn't say that it created problems, just that it wouldn't be helpful in my case (I don't need to allocate more than 32GiB rw space, so why should I set it to a lower value, not to mention its just an upper value anyway?)

Yes, 50% of available memory is the default size for tmpfs.I am not sure it can be configured in this case - that is using overlayfs with dracut.

Maybe a possible approach would be to expose a size specifier for tmpfs at boot cmdline, so the overlayfs size could be derived from that? But no idea what kind of implications that would have (if its even possible to implement), so feel free to ignore this suggestion.

tmpfs can be resized with mount -o remount,size=8G /tmp.

That works for tmpfs, but it doesn't seem to have an effect for overlayfs mounts.

But at this point it would have been just a nice-to-have feature for me anyway, so I'll just close this issue and be done with it.

leahneukirchen commented 2 years ago

Doesn't the overlay mount combine a tmpfs? Why not resize that?

Zapeth commented 2 years ago

I thought I had already tried that, but I probably just didn't use the correct mount point target.

Either way, that indeed fixes the issue for me, thanks for the tip!