search

turtlemonvh / ionic-spark-utils

Utilities for working with Ionic encryption via Spark.
MIT License
0 stars 0 forks source link

Add key caching options to dataframe transformer #2

Closed turtlemonvh closed 4 years ago

turtlemonvh commented 4 years ago

We want to re-use the keys for performance. Be sure to consider the effects of partitioning. Caching should be applied on encryption sometimes, but decryption all the time via a lightweight TTL LRU cache.

turtlemonvh commented 4 years ago

I took a lazier approach with a collection of immutable HashMap based caches (which is fine for this use case), and I'll swap that for a more performant cache layer later.