search

bottlerocket-os / bottlerocket

An operating system designed for hosting containers
https://bottlerocket.dev
Other
8.55k stars 503 forks source link

Use any of the compression algorithms supported by journlad #3317

Open arnaldo2792 opened 1 year ago

arnaldo2792 commented 1 year ago

What I'd like: journald has support for lz4, zstd, xz and zlib. Enabling one of them will improve I/O. As part of this feature, we need to be sure that the fluent-bit and cloudwatch agents support the selected compression algorithm, as well as journalctl in the admin container.

Any alternatives you've considered: Don't use compression for the journal

BinSquare commented 11 months ago

Compression is enabled by default if the compress is not set and it is not based on the current bottlerocket journald.conf settings

Ref: https://www.commandlinux.com/man-page/man5/journald.conf.5.html

It looks like it defaults to the xz compression algorithm: https://manpages.debian.org/jessie/systemd/journald.conf.5.en.html#:~:text=If%20enabled%20(the%20default)%2C,written%20to%20the%20file%20system.

@arnaldo2792 Is it reasonable that the current defaults are enough or do we suspect it's something that needs a deeper look?

arnaldo2792 commented 11 months ago

Compression is enabled by default if the compress is not set and it is not based on the current bottlerocket journald.conf settings

Compression is enabled by default only if systemd was compiled with support for any of the compression libraries, see this file. We currently don't enable support for any of the compression libraries.

Is it reasonable that the current defaults are enough or do we suspect it's something that needs a deeper look?

We need to have a deeper look into the fluent-bit and cloudwatch agents code to understand how they read the journal, otherwise these agents will fail if we enable the wrong compression algorith.