search

OSInside / kiwi

KIWI - Appliance Builder Next Generation
https://osinside.github.io/kiwi
GNU General Public License v3.0
289 stars 147 forks source link

zerofree support (which was in legacy kiwi) is missing in kiwi #2255

Closed dirkmueller closed 2 weeks ago

dirkmueller commented 1 year ago

Problem description

zerofree is a tool to zero out XFS and ext2/3/4 filesystem blocks that are no longer allocated but maybe have been written to in the past. this improves compression later somewhat.

legacy kiwi had some code to deal with this:

https://github.com/OSInside/kiwi-legacy/blob/1d4f7b139c481f063e4e008059439bdcf03ff482/modules/KIWIBoot.pm#L2734

I don't find this in the kiwi git tree and I find no plugin. how / where did that move? is there a different way to achieve this nowadays?

Expected behaviour

Steps to reproduce the behaviour

OS and Software information

schaefi commented 1 year ago

You are right this code path was not ported to the kiwi-ng version. Interesting that it pops now.

is there a different way to achieve this nowadays?

I suggest to add a patch to kiwi and add the free blocks scanner back. It would be good to add this as a capability of the filesystem class and support it for the filesystems for which such a tooling exists. imho extX and XFS only.

It would be good to get some numbers though because I don't know if there are really that many blocks to clear in the process of an image build. If your time permits could you check on one of the integration test images e.g Virtualization:Appliances:Images:Testing_x86:tumbleweed/ if you postprocess the filesystem with zerofree and how much it saves us on compressing the result ?

Conan-Kudo commented 1 year ago

We shouldn't need it in the current kiwi, since we don't create the disk image until the very end and sync the files over to it.

schaefi commented 1 year ago

That was also my understanding, but maybe @dirkmueller has a workflow in mind where it would pay off. That's why I'm waiting for his feedback too. Thanks

dirkmueller commented 1 year ago

I will do experiments and see if there's a gain. I haven't determined yet whether zerofree would bring a gain

Conan-Kudo commented 6 months ago

Do we still need this, @dirkmueller? At least from my own tests and from an incidental analysis on the Fedora side, I think we don't need this.

Conan-Kudo commented 2 weeks ago

I'm closing this issue due to lack of information and lack of apparent need of this.

dirkmueller commented 6 days ago

fine by me, thanks for the cleanup!