search

PythonNut / quark-emacs

:rocket: An incredible wonderland of code
MIT License
137 stars 6 forks source link

Improve undo tree history compression performance #105

Closed PythonNut closed 6 years ago

PythonNut commented 6 years ago

Will require adding lz4 support to auto-compression-mode.

PythonNut commented 6 years ago

Probably worth it to benchmark using hyperfine.

PythonNut commented 6 years ago 
(setq jka-compr-use-shell nil)
(add-to-list 'jka-compr-compression-info-list
             ["\\.lz4\\'"
              "lz4 compressing"   "lz4" ("-c" "-q")      ;; -c to stdout
              "lz4 uncompressing" "lz4" ("-c" "-q" "-d") ;; -d decompress
              nil           ;; append flag
              t             ;; major mode use filename without .lz4
              "\004\021\115\014"])

(add-hook 'emacs-startup-hook #'jka-compr-update)
PythonNut commented 6 years ago

Benchmarking with hyperfine suggests a 2x speedup for switching from xz (~12ms) to lz4 (~5ms).

Upper-bound savings increase with increasing file size.

PythonNut commented 6 years ago

Seems like zstd may be a better choice:

  1. About the same speed
  2. Better compression ratio
  3. Built into jka-cmpr-hook

Disadvantages:

  1. Less common
PythonNut commented 6 years ago

In fact xz may just be an outlier in terms of compression speed for small files.

Here is a benchmark table:

Compressor Time Size
gzip 3.8 ms 34375
bzip2 21.6 ms 23037
xz 62.6 ms 18500
lz4 2.7 ms 49552
zstd 3.9 ms 27791

Therefore, it seems like it makes sense to have zstd fallback to gzip.