Closed jcsp closed 4 days ago
Last week:
This week:
This week:
image_compression
config param.From @Bodobolero's benchmarks: add lz4 support for comparison.
we talked about this in the call and agreed that until further investigation in which compression is identified as culprit, we will not spend developer time on this.
I think this can be closed now.
Background
We may substantially decrease the capacity & bandwidth footprint of tenants by compressing data in their image layers.
There are many possible implementations, from compressing whole layers files as streams, to introducing some chunked format and decompressing a chunk at a time, to simply compressing individual pages.
Compressing individual pages in image layers is by far the simplest thing to do, and should have a high payoff as:
Compressing deltas is a harder problem (individual deltas are likely too small to usefully compress), and is left as a possible future change.
Implementation
There is a preliminary version here: https://github.com/neondatabase/neon/pull/7091, which demonstrates that per-page compression in image layers may be added as a relatively lightweight code change.
To get this ready for production, there is more work to do:
PRs/issues
Rollout