search

whiver / nifi-protobuf-processor

An Apache NiFi processor to encode and decode data using Google Protocol Buffer schemas.
MIT License
36 stars 20 forks source link

Error handling deleted the original content & added new Processor to convert protobuf to Avro #10

Open benyaa opened 2 years ago

benyaa commented 2 years ago

Hi, It seems that the current error handling resets the FlowFile's original content.. So:

  1. I fixed it by throwing exception so the original FlowFile won't be rewritten and then transfer it to failure.
  2. I added a capability to preserve the field names
  3. I added a capability to process messages separated by a demarcator(like when consuming using the ConsumeKafka processor)
  4. I added a new processor that converts the Protobuf messages into Avro messages so they would be easier to handle with NiFi(rather than JSON that doesn't preserve the original schema)
  5. I changed the processors' names to EncodeProtobuf and DecodeProtobuf to match NiFi conventions

Hope it's good enough!