well-typed / grapesy

Native Haskell gRPC client and server based on `http2`
Other
31 stars 4 forks source link

Default message size threshold for compression #144

Closed FinleyMcIlwaine closed 1 month ago

FinleyMcIlwaine commented 1 month ago

We used to always compress outgoing messages if the compression was not identity and the compressed length was less than the uncompressed length. Now, we allow the user to configure a length threshold for uncompressed messages for compression to be considered, per algorithm. The default threshold for each algorithm is the length for which a message of all zeroes achieves a 8:7 compression ratio. This 8:7 ratio was somewhat arbitrarily picked as a target ratio.

To reproduce the ratio tests I did for the compression algorithms, in a cabal repl on grapesy (replace gzip with whichever other compression scheme you want to test):

import Codec.Compression.GZip qualified as GZip
import Data.ByteString.Lazy qualified as BS.Lazy
ratio bs = (fromIntegral $ BS.Lazy.length bs) / (fromIntegral $ BS.Lazy.length (GZip.compress bs)) :: Double
results = [ (size, rat) | size <- [0 ..], let rat = ratio (BS.Lazy.replicate size 0) ]
takeWhile (\(_, rat) -> rat < (1 / 0.875)) results