Open turtlemonvh opened 4 years ago
In addition to these options, I need to extend the parameters for the transformer to take more complex configuration for each action, so we can write transformed columns to custom named columns instead of requiring a rename every time.
A nice interface to provide here would be
// Before
def Encrypt(encryptCols: List[String], decryptCols: List[String], agentFactory: () => KeyServices)(df: DataFrame): DataFrame
// After
def Encrypt(actions: List[IonicAction], agentFactory: () => KeyServices)(df: DataFrame): DataFrame
Where IonicAction
is a type we define that defines the action to perform, which gives us flexibility to extend capabilities to deterministic encryption (see #7 ) and FPE (see #11 ), and also makes it easy to add custom metadata, attributes, attributes derived from data, column renaming, and even data types (e.g. binary vs string).
The IonicAction
also gives us a good interface for testing, since we'll be able to test
IonicAction
sIonicAction
sI also plan to support nested columns here.
This is likely best configured via lambda functions.
With all these options, it's also time to change the transform function from taking a simple parameter list to taking a case class options object (e.g.
IonicEncryptTranformOptions
).