Adds a transformation package and integration into the main functions.
The design is intended to be generalisable to applying any kind of transformation upon any kind of data (including filters), but this PR only provides an implementation for enriched Snowplow events' transformation to JSON.
Three things I'll point out as being uncertain of (mostly documented inline also):
We edit the input when we perform a transformation (I believe because messages takes a pointer?)
We could construct new messages objects, but this would be tricky with the performance metrics
I'm unsure if this is acceptable, or if I should dereference pointers or something?
The unit tests are tricky because of point 1. I'm unsure if some of the equivalence checks are actually a tautological pass because the input is edited by the transformation function.
The specific implementations of transformations don't currently return any error. The assumption here is that any error we hit would result in a message going to the Invalid slice in the result (which contains an error message).
Adds a transformation package and integration into the main functions.
The design is intended to be generalisable to applying any kind of transformation upon any kind of data (including filters), but this PR only provides an implementation for enriched Snowplow events' transformation to JSON.
Three things I'll point out as being uncertain of (mostly documented inline also):
The unit tests are tricky because of point 1. I'm unsure if some of the equivalence checks are actually a tautological pass because the input is edited by the transformation function.
The specific implementations of transformations don't currently return any error. The assumption here is that any error we hit would result in a message going to the Invalid slice in the result (which contains an error message).