search

logzio / sawmill

Sawmill is a JSON transformation Java library
Apache License 2.0
116 stars 24 forks source link

add base64 decode processor #311

Closed ronshay closed 1 year ago

ronshay commented 1 year ago

This processor decodes base64 encoded values

ronshay commented 1 year ago
  1. Just curious, why do we provide the processor on a list of fields? The common configuration of most processors usually gets one "source field" and one "target field". If you want to apply a processor on multiple fields, you need to define multiple processors/steps in your pipeline.
  2. I see you decided to change the value in-place (override the original field value) rather than setting a "target field". I wonder if this will make things harder to debug later, if some processing outputs bad decoding. Another option is to leave the original field as is, and if the user wants to override it then he just needs to add removeField and renameField processors to do that. If something goes wrong, he can omit the two additional steps for debugging.

@DanMelman nice points. regarding the first one, i figured a single processor on multiple fields will probably run faster than n processors, but i do think it's a valid option.

about your second point, i totally agree. that will allow us to easily debug and the user can remove/rename as needed. so i think we can go with keeping the original field, and setting a new field (target) with the decoded value. WDYT?