sstable: use github.com/klauspost/compress for snappy

[jbowens]

Use the github.com/klauspost/compress module's Snappy implementation. It's faster (10-20%) and generates comparable compressed sizes. Although it obeys the same format as Snappy and is bi-directionally compatible with Google's Snappy implementation, it does not produce identical payloads (claiming to produce slightly smaller payloads).

                                                                                │   old.txt    │               new.txt                │
                                                                                │     B/s      │     B/s       vs base                │
Writer/format=(Pebble,v2)/block=4.0KB/filter=true/compression=NoCompression-24    297.7Mi ± 0%   298.4Mi ± 1%        ~ (p=0.093 n=10)
Writer/format=(Pebble,v2)/block=4.0KB/filter=true/compression=Snappy-24           70.44Mi ± 1%   62.62Mi ± 0%  -11.10% (p=0.000 n=10)
Writer/format=(Pebble,v2)/block=4.0KB/filter=true/compression=ZSTD-24             13.66Mi ± 1%   13.67Mi ± 1%        ~ (p=0.698 n=10)
Writer/format=(Pebble,v2)/block=4.0KB/filter=false/compression=NoCompression-24   421.3Mi ± 0%   420.3Mi ± 0%   -0.23% (p=0.050 n=10)
Writer/format=(Pebble,v2)/block=4.0KB/filter=false/compression=Snappy-24          85.63Mi ± 0%   72.42Mi ± 1%  -15.42% (p=0.000 n=10)
Writer/format=(Pebble,v2)/block=4.0KB/filter=false/compression=ZSTD-24            12.09Mi ± 1%   12.22Mi ± 1%        ~ (p=0.108 n=10)
Writer/format=(Pebble,v2)/block=32KB/filter=true/compression=NoCompression-24     304.1Mi ± 0%   306.4Mi ± 0%   +0.74% (p=0.000 n=10)
Writer/format=(Pebble,v2)/block=32KB/filter=true/compression=Snappy-24            55.25Mi ± 0%   46.94Mi ± 0%  -15.05% (p=0.000 n=10)
Writer/format=(Pebble,v2)/block=32KB/filter=true/compression=ZSTD-24              15.60Mi ± 1%   15.70Mi ± 2%        ~ (p=0.197 n=10)
Writer/format=(Pebble,v2)/block=32KB/filter=false/compression=NoCompression-24    437.5Mi ± 0%   435.8Mi ± 0%   -0.38% (p=0.000 n=10)
Writer/format=(Pebble,v2)/block=32KB/filter=false/compression=Snappy-24           65.57Mi ± 0%   52.02Mi ± 0%  -20.66% (p=0.000 n=10)
Writer/format=(Pebble,v2)/block=32KB/filter=false/compression=ZSTD-24             12.07Mi ± 1%   12.09Mi ± 1%        ~ (p=0.254 n=10)
Writer/format=(Pebble,v3)/block=4.0KB/filter=true/compression=NoCompression-24    281.1Mi ± 0%   282.1Mi ± 0%   +0.34% (p=0.023 n=10)
Writer/format=(Pebble,v3)/block=4.0KB/filter=true/compression=Snappy-24           65.62Mi ± 0%   58.64Mi ± 0%  -10.65% (p=0.000 n=10)
Writer/format=(Pebble,v3)/block=4.0KB/filter=true/compression=ZSTD-24             12.97Mi ± 2%   13.17Mi ± 1%   +1.47% (p=0.019 n=10)
Writer/format=(Pebble,v3)/block=4.0KB/filter=false/compression=NoCompression-24   387.8Mi ± 0%   388.9Mi ± 0%   +0.27% (p=0.005 n=10)
Writer/format=(Pebble,v3)/block=4.0KB/filter=false/compression=Snappy-24          78.29Mi ± 0%   67.37Mi ± 0%  -13.95% (p=0.000 n=10)
Writer/format=(Pebble,v3)/block=4.0KB/filter=false/compression=ZSTD-24            11.46Mi ± 3%   11.71Mi ± 2%   +2.16% (p=0.000 n=10)
Writer/format=(Pebble,v3)/block=32KB/filter=true/compression=NoCompression-24     285.8Mi ± 0%   287.8Mi ± 0%   +0.68% (p=0.000 n=10)
Writer/format=(Pebble,v3)/block=32KB/filter=true/compression=Snappy-24            51.15Mi ± 0%   43.94Mi ± 0%  -14.10% (p=0.000 n=10)
Writer/format=(Pebble,v3)/block=32KB/filter=true/compression=ZSTD-24              14.77Mi ± 1%   14.93Mi ± 0%   +1.03% (p=0.001 n=10)
Writer/format=(Pebble,v3)/block=32KB/filter=false/compression=NoCompression-24    402.4Mi ± 0%   403.0Mi ± 0%        ~ (p=0.123 n=10)
Writer/format=(Pebble,v3)/block=32KB/filter=false/compression=Snappy-24           59.84Mi ± 0%   48.51Mi ± 0%  -18.93% (p=0.000 n=10)
Writer/format=(Pebble,v3)/block=32KB/filter=false/compression=ZSTD-24             11.42Mi ± 1%   11.56Mi ± 1%   +1.29% (p=0.000 n=10)
geomean                                                                           66.51Mi        63.24Mi        -4.91%

[cockroach-teamcity]

This change is 

[petermattis]

Ditto on the nice find! Is the speedup also present on Arm? Did you also look at decompression speed?

[jbowens]

There's an issue here that the library doesn't seem to produce the same output on all platforms, and our tests depend on determinism of compression across platforms. I will return to this and try to dig a little deeper to where the platform dependence is coming from—it seems surprising and undesirable.

[RaduBerinde]

Interesting.. What platform did you use to generate them? If it was MacOS, I would expect the go-macos test to pass and others to fail?

[jbowens]

Yeah, I generated locally on my arm mac. I think it must be arch dependent, and the go-macos action uses an intel mac