The NiFiKop NiFi Kubernetes operator makes it easy to run Apache NiFi on Kubernetes. Apache NiFI is a free, open-source solution that support powerful and scalable directed graphs of data routing, transformation, and system mediation logic.
I am seeking guidance on best practices for efficiently managing shared data in the NiFi Kops environment. Currently, I am considering several approaches and would greatly appreciate your insights.
The proposed approach involves uploading the required JAR file directly onto the pod. However, I believe there might be more robust strategies to handle shared data effectively.
After some research, I am contemplating the following best practices:
Implementing a Shared Storage Solution:
Creating a shared storage solution using Persistent Volumes (PV) and Persistent Volume Claims (PVC) via Container Storage Interface (CSI) is a recommended approach. By adopting this method, we can ensure data persistence and enable seamless sharing of data among pods.
Data Loading Mechanism:
To populate the shared volume with relevant data, I intend to utilize an external script that automates the data loading process.
Volume Mounting for Pods:
By effectively mounting the shared volume to all relevant pods, we can ensure consistent access to the shared data across the NiFi Kops deployment. Properly configuring the mounting process is essential to achieve optimal performance and data integrity.
Exploring Cloud Blob Storage Integration:
As part of our investigation into optimal shared data practices, we are also exploring the possibility of leveraging cloud-based blob storage services such as Amazon S3 or Azure Blob Storage. This approach might provide additional benefits in terms of scalability, data redundancy, and cost-effectiveness.
In conclusion, I kindly request the experienced members of the NiFi Kops community to share their valuable insights and recommendations regarding the above-mentioned strategies.
Type of question
Best Practices
Support question
Dear NiFi Kops community,
I am seeking guidance on best practices for efficiently managing shared data in the NiFi Kops environment. Currently, I am considering several approaches and would greatly appreciate your insights.
The proposed approach involves uploading the required JAR file directly onto the pod. However, I believe there might be more robust strategies to handle shared data effectively.
After some research, I am contemplating the following best practices:
Implementing a Shared Storage Solution:
Exploring Cloud Blob Storage Integration:
As part of our investigation into optimal shared data practices, we are also exploring the possibility of leveraging cloud-based blob storage services such as Amazon S3 or Azure Blob Storage. This approach might provide additional benefits in terms of scalability, data redundancy, and cost-effectiveness.
In conclusion, I kindly request the experienced members of the NiFi Kops community to share their valuable insights and recommendations regarding the above-mentioned strategies.
Thank you for your time and consideration.
NiFiKop version
No response
Golang version
No response
Kubernetes version
No response
NiFi version
No response