Add NiFi cluster default settings

[r65535]

Feature Request

Is your feature request related to a problem? Please describe.

NiFi now allows users to specify the following settings at the cluster level:

1. Default FlowFile Expiration
2. Default Back Pressure Object Threshold
3. Default Back Pressure Data Size Threshold

Describe the solution you'd like to see

New fields adding into the NiFiCluster CRD to set these three new properties

[mh013370]

I see nifi.queue.backpressure.count and nifi.queue.backpressure.size in nifi.properties to change the default backpressure object and data size settings. However, i don't see a way to change the default flowfile expiration at the cluster-level. Am i just overlooking it?

https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html

[r65535]

I don't think it's been added to their admin guide... You can set all three properties in the UI, which then sends a PUT request to /nifi-api/process-groups/root with this kind of JSON:

{
   "revision":{
      "clientId":"d415e910-0182-1000-442a-86cff9363a1e",
      "version":0
   },
   "disconnectedNodeAcknowledged":false,
   "component":{
      "id":"d015817d-0182-1000-b48c-d6e743c6b26d",
      "name":"NiFi Flow",
      "comments":"",
      "parameterContext":{
         "id":null
      },
      "flowfileConcurrency":"UNBOUNDED",
      "flowfileOutboundPolicy":"STREAM_WHEN_AVAILABLE",
      "defaultFlowFileExpiration":"1 sec",
      "defaultBackPressureObjectThreshold":"10000",
      "defaultBackPressureDataSizeThreshold":"1 GB"
   }
}

[mh013370]

Ah okay so perhaps this would be an addition to a NifiDataflow to include defaults for these properties. Then if the properties aren't explicitly configured in a versioned flow in Registry, then the defaults configured in the NifiDataflow would take precedence.

Does that seem reasonable?

[r65535]

Good point - these settings are on a processGroup level. The root processGroup settings are propagated down when a nested processGroup is added, but can be overridden. Should we allow for both - one setting in NiFiCluster to set the default and then optional fields in NiFiDataflow to override?

[juldrixx]

I ask this without having tested it, but won't these parameters be considered as changes in the versioning of the dataflows and cause repeated rollbacks of them?