Updating ZSTD to latest release in Liquorix

[anonyco]

The current ZSTD version in Liquorix is currently the same as in the upstream kernel—1.5.2. Zstd has made numerous leaps and jumps in performance over the past few years. Some highlights from their release notes:

• Among the 1.5.4 release notes: platform	corpus	v1.5.2	v1.5.4-no-asyncio	v1.5.4	Improvement
  Xeon D-2191A CentOS8	enwik8	280 MB/s	280 MB/s	324 MB/s	+16%
  Xeon D-2191A CentOS8	silesia.tar	303 MB/s	302 MB/s	386 MB/s	+27%
  i7-1165g7 win10	enwik8	270 MB/s	280 MB/s	350 MB/s	+27%
  i7-1165g7 win10	silesia.tar	450 MB/s	440 MB/s	580 MB/s	+28%
  i7-9700K Ubuntu20	enwik8	600 MB/s	604 MB/s	829 MB/s	+38%
  i7-9700K Ubuntu20	silesia.tar	683 MB/s	678 MB/s	991 MB/s	+45%
  Galaxy S22	enwik8	360 MB/s	420 MB/s	515 MB/s	+70%
  Galaxy S22	silesia.tar	310 MB/s	320 MB/s	580 MB/s	+85%
  MBP M1	enwik8	428 MB/s	734 MB/s	815 MB/s	+90%

• Among the v1.5.5 release notes:	sample size	v1.5.4 (MB/s)	v1.5.5 (MB/s)	improvement
  100	1.4	44.8	x32
  200	2.8	44.9	x16
  500	6.5	60.0	x9.2
  1K	12.4	70.0	x5.6
  2K	25.0	111.3	x4.4
  4K	44.4	139.4	x3.2
  ...	...	...
  1M	97.5	99.4	+2%

• Among the v1.5.6 release notes. This is particularly interesting for speeding up btrfs:

  …offers some interesting latency reduction, notably beneficial in areas with less powerful network infrastructure.…

[damentz]

Do you have a link to the updated kernel implementation, preferably a patch or branch I can merge?

[Rongronggg9]

IMHO these snippets are just cherry-picking, and even the changelog itself from ZSTD did so. I can hardly see any potential that it may speed BTRFS up.

Among the 1.5.4 release notes: [...]

Before the table, it says: The 1.5.4 release improves IO performance of zstd CLI...

So it is irrelevant to libzstd.

Among the v1.5.5 release notes: [...]

Before the table, it says: The row-hash match finder introduced in version 1.5.0 for levels 5-12 has been improved in version 1.5.5... _The following scenario measures compression speed of ZSTD_compressStream at level 9..._

BTRFS users hardly use such expensive levels (5-12) and the optimization is irrelevant to lower levels. Meanwhile, the compression of BTRFS is done chunk-by-chunk (max 128KiB). According to the table, it is suspicious that a chunk as large as 128KiB would see a notable gain. As for small chunks, BTRFS will find the overhead is just too high and stop compressing them.

Among the v1.5.6 release notes. This is particularly interesting for speeding up btrfs: [...]

Before the citation, it says: _...libzstd introduces the new parameter ZSTD_c_targetCBlockSize, ... Activating this feature incurs a cost, both runtime (equivalent to -2% speed at level 8) and a slight compression efficiency decrease (<0.1%), but offers some interesting latency reduction, notably beneficial in areas with less powerful network infrastructure._

So it is an opt-in feature and is not a silver bullet. Just bumping the version will not take any advantage of it. Meanwhile, I can hardly see SSDs (even HDDs - typical seek time needs just 4-15ms!) are comparable to "areas with less powerful network infrastructure".

[damentz]

@Rongronggg9 thanks for the analysis, it looks like users won't notice the benefit. Not to mention, Liquorix defaults to LZ4 wherever possible to minimize compute time of compression.

Closing this out.