search

projg2 / eclean-kernel

Installed kernel cleanup tool
GNU General Public License v2.0
32 stars 11 forks source link

eclean-kernel has met the following issue: MissingDecompressorError('Kernel file /boot/initramfs-5.13.12-gentoo-x86_64.img is compressed with lzo, but the required decompressor is not installed') #22

Open iq180 opened 3 years ago

iq180 commented 3 years ago

However, emerge --info emerge app-arch/lzop shows:

...
=================================================================
                        Package Settings
=================================================================

app-arch/lzop-1.04::gentoo was built with the following:
USE="" ABI_X86="(64)"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs ccache clean-logs config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
mgorny commented 3 years ago

There's USE=lzo that pulls in the right deps.

iq180 commented 3 years ago

What do you mean?

My kernel and initramfs are compiled using LZO compression.

I already have an installed package "app-arch/lzop".

I've added "app-admin/eclean-kernel lzo" to my "/etc/portage/package.use", re-emerged "app-admin/eclean-kernel", and now when I run "eclean-kernel -n 1", I get a message: "error('Header error - invalid compressed data')".

iq180 commented 3 years ago 
# modprobe configs && zgrep LZO /proc/config.gz
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_KERNEL_LZO=y
CONFIG_RD_LZO=y
# CONFIG_CRYPTO_LZO is not set
CONFIG_LZO_DECOMPRESS=y
CONFIG_DECOMPRESS_LZO=y
# emerge --info eclean-kernel
...
app-admin/eclean-kernel-2.99.3::gentoo was built with the following:
USE="lzo -lz4 -test -zstd" ABI_X86="(64)" PYTHON_TARGETS="python3_9 -python3_8"
FEATURES="userfetch protect-owned xattr assume-digests usersync strict config-protect-if-modified parallel-fetch ebuild-locks network-sandbox unknown-features-warn qa-unresolved-soname-deps sandbox unmerge-orphans binpkg-multi-instance sfperms userpriv pid-sandbox ipc-sandbox binpkg-logs ccache clean-logs unmerge-logs fixlafiles news binpkg-dostrip multilib-strict fail-clean binpkg-docompress merge-sync usersandbox distlocks preserve-libs"
# eclean-kernel -n 1
eclean-kernel has met the following issue:

  error('Header error - invalid compressed data')

If you believe that the mentioned issue is a bug, please report it
to https://github.com/mgorny/eclean-kernel/issues. If possible,
please attach the output of 'eclean-kernel --list-kernels' and your
regular eclean-kernel call with additional '--debug' argument.
mgorny commented 3 years ago 
emerge -pv python-lzo

please.

mgorny commented 3 years ago

CC @Jannik2099

iq180 commented 3 years ago 
emerge -pv python-lzo

please.

It was emerged as a dependency of eclean-kernel.

-rw-rw---- 1 portage portage 4,1K 2021-08-20 01:38 /var/log/portage/dev-python:python-lzo-1.12-r2:20210819-223809.log
-rw-rw---- 1 portage portage 8,0K 2021-08-20 01:38 /var/log/portage/app-admin:eclean-kernel-2.99.3:20210819-223829.log
-rw-rw---- 1 portage portage  20K 2021-08-20 01:38 /var/log/portage/app-admin:eclean-kernel-2.99.3:20210819-223820.log
Jannik2099 commented 3 years ago

well this is interesting

can you open a python shell (just run python) and then run import lzo ? Does python-lzo have the same PYTHON_TARGETS as eclean-kernel?

@mgorny I wasn't aware we also do the kernel version parsing on initramfs files, is this intended?

iq180 commented 3 years ago

can you open a python shell (just run python) and then run import lzo ?

# python
Python 3.9.6 (default, Aug  5 2021, 20:29:37) 
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>> import lzo
>>

Does python-lzo have the same PYTHON_TARGETS as eclean-kernel?


# emerge --info|grep PYTHON
USE="...PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9"..."

emerge --info python-lzo|grep PYTHON

... USE="-test" ABI_X86="(64)" PYTHON_TARGETS="python3_9 -python3_10 -python3_8"

emerge --info eclean-kernel|grep PYTHON

... USE="lzo -lz4 -test -zstd" ABI_X86="(64)" PYTHON_TARGETS="python3_9 -python3_8"

mgorny commented 3 years ago

@Jannik2099 see https://github.com/mgorny/eclean-kernel/issues/22#issuecomment-902303114. Maybe the kernel uses a different stream format, similarly to lz4?

@iq180 would it be possible for you to attach the kernel image here? I don't think there are any secrets inside it ;-).

Jannik2099 commented 3 years ago

ooooh, I didn't see that the error changed - whoops Yeah, it's probably omitting some metadata then... why you do this kernel :(

Jannik2099 commented 3 years ago

https://www.kernel.org/doc/Documentation/lzo.txt oh great, what a fustercluck

mgorny commented 3 years ago

If it's too hard to support, we can always declare it unsupported. Would be nice to get an lzo-compressed kernel anyway and see if lzop can deal with it.

Jannik2099 commented 3 years ago

yeah, if this requires any custom logic I'd prefer to just drop it - lzo is obsoleted by zstd anyways in just about any metric (including being a sane format) I'll try reproduce this the next few days

iq180 commented 3 years ago

yeah, if this requires any custom logic I'd prefer to just drop it - lzo is obsoleted by zstd anyways in just about any metric (including being a sane format) I'll try reproduce this the next few days

Hm. LZO is fast, it's approved to use in kernel/initramfs compression. But "it's too hard to support". Very strange.

Apparently, eclean is for those who use default kernel compilation settings only.

Very strange.

iq180 commented 3 years ago

If it's too hard to support, we can always declare it unsupported

Great decision.

Jannik2099 commented 3 years ago

it's approved to use in kernel/initramfs compression. But "it's too hard to support". The kernel uses a custom encoding, not a widely standardized one

LZO has roughly the same compression ratio as lz4 at almost 5x lower decompression speed, and a roughly 30% lower compression ratio than zstd at less than half the decompression speed. It's outclassed by lz4 and zstd at both ends https://facebook.github.io/zstd/

That being said if it turns out python-lzo supports the kernels compression encoding there's no reason not to support it, but I don't feel like writing a custom frame decompressor for an effectively deprecated algorithm

mgorny commented 3 years ago

@iq180, please stop being such an asshole. We're working on this in our free time, we don't get any money for it and apparently no gratitude either. We can't do much about the fact that LZO is poorly standardized and the kernel implementation apparently disagrees with the standard stream format.

If you want things to happen, either supply patches or talk to @Jannik2099 about the possibility of funding this work. Or at least stop being such a douchebag because that kind of attitude is literally killing volunteer work.

mgorny commented 3 years ago

@Jannik2099 BTW you may want to see if squashfs-tools has something useful to steal. If you really don't want to do it, then could you remove LZO support altogether and replace it with a clear error message?

iq180 commented 3 years ago

@iq180, please stop being such an asshole. We're working on this in our free time, we don't get any money for it and apparently no gratitude either. We can't do much about the fact that LZO is poorly standardized and the kernel implementation apparently disagrees with the standard stream format.

If you want things to happen, either supply patches or talk to @Jannik2099 about the possibility of funding this work. Or at least stop being such a douchebag because that kind of attitude is literally killing volunteer work.

Asshole, as I consider, is the maintainer who fails to maintain.

eternal-sorrow commented 5 months ago

Why does it need to decompress initramfs images though? Can I somehow tell it to ignore them?