search

divviup / janus

Experimental implementation of the Distributed Aggregation Protocol (DAP) specification.
Mozilla Public License 2.0
51 stars 14 forks source link

Disable TOAST compression for encrypted columns #1212

Open divergentdave opened 1 year ago

divergentdave commented 1 year ago

Several columns in our schema will hold possibly-large encrypted VDAF messages. We should disable compression on these columns when they are created to avoid wasted work. Most of these VDAF messages will be indistinguishable from randomness, and that's before we encrypt them with AES-GCM! It appears that by default, columns with no COMPRESSION selected would be compressed when TOASTed.

SHOW default_toast_compression;
 default_toast_compression 
---------------------------
 pglz
(1 row)
branlwyd commented 1 year ago

👍🏻 to the idea, though I wouldn't prioritize this too quickly, for the following reasons: (a) Postgres' compression scheme (pglz) is designed to be CPU-cheap and (b) TOAST only kicks in for values over 2 KiB, which I expect our encrypted columns will not normally reach. (Perhaps we'd reach 2 KiB for some of our encoded VDAF messages? In that case, only argument (a) remains.)