search

GoogleCloudPlatform / flink-on-k8s-operator

[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.
Apache License 2.0
658 stars 266 forks source link

Enable High Availability for Flink job manager on k8 cluster #358

Closed shravangit20 closed 3 years ago

shravangit20 commented 3 years ago

Hi,

We have just migrated to Kubernetes (EKS) and setting up the Flink cluster/operator on the K8s at the moment. We need to enable HA for the flink job manager and since we already have an AWS MSK (AWS managed kafka which is on zookeeper) we may not want to setup another zookeeper cluster on EKS (Kubernetes).

Just wanted to check if there are any alternatives to zookeeper which can work for HA ? I see there is a native kubernetes HA service is being developed and is in-progress but since we want to implement the solution immediately want to know if there is a stable solution available.

Thanks, Shravan

shravangit20 commented 3 years ago

Can anyone please help?

functicons commented 3 years ago

@shravangit20, the Flink operator does't support native Kubernetes HA yet and there is no plan to support it in the near future. For now, you have to do it the old way with ZooKeeper I think.

shravangit20 commented 3 years ago

Thank you @functicons