cetic / helm-nifi

Helm Chart for Apache Nifi
Apache License 2.0
215 stars 225 forks source link

Load custom nar #156

Closed bmgante closed 3 years ago

bmgante commented 3 years ago

I am trying to load a custom NAR using the helm chart and following the indications from readme. However i have some doubts if after creating this properties and extra volumes how to load the nar itself. Should i manually load it to the container directrory using kubectl or is there any way to store the nar in the project and the chart automatically transfers it to the container when deploying?

extraVolumeMounts:
    - name: mycustomlibs
      mountPath: /opt/configuration_resources/custom_lib
  extraVolumes: # this will create the volume from the directory
    - name: mycustomlibs
      hostPath:
        path: "CUSTOM_LIB_FOLDER"
  properties:
    customLibPath: "/opt/configuration_resources/custom_lib"
bmgante commented 3 years ago

I was able to get the custom processor after transferring the nar to the CUSTOM_LIB_FOLDER i´ve defined for each of cluster nifi container. However if for some reason pod restarts, the nar does not persists. Is it possible to enable persistence as well for this purpose?

bmgante commented 3 years ago

Indeed, not matter of the path i define for CUSTOM_LIB_FOLDER i am only able to get the custom processor available in UI if transferring the nar to the existing extensions directory:

kubectl -n xxx cp Downloads/nifi-xxx-nar-1.0.9.nar nifi-0:/opt/nifi/nifi-current/extensions -c server  --kubeconfig=$HOME/.kube/config-files/xxx.yml
kubectl -n xxx cp Downloads/nifi-xxx-nar-1.0.9.nar nifi-1:/opt/nifi/nifi-current/extensions -c server  --kubeconfig=$HOME/.kube/config-files/xxx.yml
kubectl -n xxx cp Downloads/nifi-xxx-nar-1.0.9.nar nifi-2:/opt/nifi/nifi-current/extensions -c server  --kubeconfig=$HOME/.kube/config-files/xxx.yml

After transferring the nar need to full refresh UI page or even open a new session in order to get the nar available in the list of processors. Does anyone can advise if this is the right procedure to get custom NAR available on nifi helm chart?

bmgante commented 3 years ago

My config:

  extraVolumeMounts:
    - name: mycustomlibs
      mountPath: /opt/configuration_resources/custom_lib
  extraVolumes: # this will create the volume from the directory
    - name: mycustomlibs
      hostPath:
        path: "/opt/nifi/data/custom_lib"
  properties:
    customLibPath: "/opt/configuration_resources/custom_lib"
banzo commented 3 years ago

Some suggestions here: https://stackoverflow.com/questions/60280381/helm-bitnami-spark-how-to-load-files-to-extravolumemounts

bmgante commented 3 years ago

Thanks @banzo. I set up custom libs path to a pvc volume and it works fine as well.

umar224485 commented 3 years ago

@bmgante how did you setup custom libs path to pvc. Can you share the changes I am facing same issue

bmgante commented 3 years ago

Just set the path for pvc in customLibPath parameter.

eg: customLibPath: "/opt/nifi/data/custom_libs"

umar224485 commented 3 years ago

@bmgante Thanks, I have my values as below customLibPath: "/opt/configuration_resources/custom_lib" (this is under properties) and

extraVolumeMounts:

I have my nar files at /home/ubuntu/nars but once I run "helm install nifi ." the nar files are not getting copied to the pod.

Do I need to manually copy the nar files or am I missing something?

bmgante commented 3 years ago

Indeed, I am copying them manually to the PVC...

github-actions[bot] commented 3 years ago

This issue is stale because it has not seen recent activity. Remove stale label or comment or this will be closed.