search

koverstreet / bcachefs

Other
700 stars 73 forks source link

Slow compressed writes and high CPU usage #771

Open brauliobo opened 1 month ago

brauliobo commented 1 month ago

On rsyncing a backup on slow WD 5TB external drives, I see a one-thread flush operation almost always on 100% CPU

The compression used is zstd:10 and zstd:15 for another drive.

On BTRFS, using the compression I see an eventual flush with a multi-threaded CPU use on compressed-write kthreads. It usually lasts just a few seconds instead of constant 1 core use.

Also, it seems Bcachefs isn't testing the compression and is dumbly compressing everything even if it wouldn't result in any gains. For instance, media (audio, video, and images) aren't compressible, besides the obvious already compressed files.

So a lot of CPU usage can be saved if it only compresses what can be compressed.

Using Linux 6.11.3 on Archlinux image

koverstreet commented 3 weeks ago

Yeah, multithreaded compression is on the wishlist; filetype detection would be good too