search

bluehalo / nifi-nar-bundles

Apache License 2.0
13 stars 0 forks source link

Add DelayAndRetry Processor #44

Closed brianghig closed 2 years ago

brianghig commented 6 years ago

Frequently for remote service requests, we create a series of processors to retry a FlowFile up to a number of times with a delay between each retry. At the moment, we're limited to implementing this with the following processor sequence:

  1. UpdateAttribute to increment the custom "retry" attribute
  2. RouteOnAttribute to detect whether the "retry" attribute has met some threshold, routing to "max.retries" if it has or "unmatched" if it has not
  3. DelayProcessor to hold the FlowFile for some period of time before the follow-on request is made

All of these could potentially be collapsed into a single processor with configurations for the Retry Count attribute name, the Max Number of Retries, and the Delay Time, significantly reducing flow construction time and the number of processors that each NiFi instance has to manage.

rstjohn commented 2 years ago

This processor has been created and is called RetryCounter. While this is included here, Apache NiFi has included similar functionality directly within relationships in version 1.16.x.