Percona-Lab / pg_tde

MIT License
107 stars 19 forks source link

Performance / memory usage fix: do not allocate memory in pg_tde_slot #265

Closed dutow closed 1 month ago

dutow commented 1 month ago

Until now, for some reason we allocated memory for each decrypted tuple in pg_tde_slot, and only freed all that memory when the transaction ended. This caused two issues:

This allocation is most likely a leftover from before we even used slots, and handled selects differently. With slots, we don't need them at all, as slots are not expected to handle multiple tuples at the same time, only the last one.

This commit removes the palloc completely, and instead adds a single BLCKSZ array to the slot structure, which can hold any size of decrypted tuple.

To be safe, it also disables the get tuple function, forcing the core code to use copy instead when needed.

This change results in:

  1. no "memory spike" during sequential scans
  2. ~1.55x overhead instead of ~2.2x

(The TODO in slot_copytuple will be addressed in a separate commit)

codeforall commented 1 month ago

Overall, the PR looks good. Just one minor comment. I believe we can now get rid of TdeSlotForgetDecryptedTuple function as we do not need to pfree the decrypted_tuple