Pulsar Operator On Kubernetes

[sky-big]

Add pulsar operator for kubernetes(https://github.com/sky-big/pulsar-operator) based on deployment/kubernetes/*.yaml

[sky-big]

Welcome everyone to contribute

[sky-big]

Basically realized the function of pulsar operator，include pulsar dashboard, grafana, prometheus and so on Pulsar Operator，weclome to use and contribute.

[sky-big]

Next step concern pulsar stateful component(include Bookeeper and Zookeeper), use operator control complex stateful component.

[chrsoo]

Great initiative! I know you have to start somewhere and cannot do everything at once but have you thought about CRD's for Sources and Sinks? What about configuration like Channels, Tenants and Namespaces?

I would really like to get out of an imperative approach based on pulsar-admin for managing the cluster. Still trying to figure out how to get the whole configuration for a geo-replicated cluster in Git...

[wolfstudy]

I would really like to get out of an imperative approach based on pulsar-admin for managing the cluster.

@chrsoo Maybe the pulsar-manager is a good idea for you.

[chrsoo]

@wolfstudy thanks for the tip (I am aware) but how is pulsar-manager not imperative?

A web UI where you do point-and-click is virtually the same thing as issuing commands on the command line. At each instance in time you tell the system what to do instead of telling it the desired target state and have it figure out the changes by itself.

In the end I would like to declare the state of my cluster in an external git repository and then have a CI/CD pipeline provision it for me (gitops).

The CRD proposed by @sky-big is a really good start and I could work with this towards my end goal, but I need to declare other parts of the configuration as well. Point-and-click will not do the job :-)

[sky-big]

@chrsoo Cloud you please give a sepcific example based on your sence? then we can add this function in the pulsar operator.

[chrsoo]

The difficulty here is where to draw the line between more or less static configuration and what should remain dynamically defined at runtime by the clients/users of the system. This might vary in different contexts.

I think it could be useful to define CRDs for the following Pulsar configuration items:

1. Tenants with namespaces and their relation to Clusters
2. Persistent Topics and their relation to tenants/namespaces
3. Sources and Sinks and their relation to clusters/topics

Starting with Tenants the following CRDs might be useful

• PulsarTenant - defines a Pulsar Tenant
• PulsarNamespace - defines a Pulsar Namespace for a Tenant
• PulsarNamespaceBinding - binds a Pulsar Namespace to a PulsarCluster

It would be good if Tenants and Namespaces are defined at the K8s cluster level as a Tenants and Namespaces then could be reused across multiple PulsarClusters defined in the different K8s namespaces. The PulsarNamespaceBinding should be defined at the namespace level and in the same namespace as the PulsarCluster to which it is bound.

@sky-big - does all this make sense to you?

[sky-big]

@wolfstudy @sijie @jiazhai Can you disscuss it together?

[sky-big]

@chrsoo I generally understand what you mean, Starting with Tenants CRDs, Can you make a specific design and let everyone discuss？

[sijie]

@chrsoo @sky-big @wolfstudy I think what @chrsoo request is an operator for the full lifecycle management for all the resources within a Pulsar instance. This includes both ClusterOperator and ResourceOperator.

A ClusterOperator is what @sky-big is doing and most of the operators are doing. It is used for setting up a cluster and managing the upgrade, downgrade and scaling up-and-down a cluster.

A ResourceOperator is more about using CRD for provisioning resources within a Pulsar cluster. A resource such as a role (key and tokens), a tenant, a namespace, a topic, a function and etc.

Once everything is modeled as CRD, you can use kubernetes to provision both cluster and resources. Thus you can apply a GitOps patten in a CI/CD pipeline to manage everything.

@chrsoo : this is kind of aligned with what we (StreamNative) are working on. We shall be able to share what we have done soon.

[chrsoo]

@sijie yes, exactly. Should we take your comment regarding StreamNative to mean that you are planning to open source an Operator for Pulsar on Kubernetes that deals with the full scope of CRD's for Pulsar management? How soon is "soon"?

@sky-big regarding a specific design I would design it as explained above but I will see if I can produce a more detailed design specification as a PR.

[sky-big]

@chrsoo OK, Looking forward to you PR.

[sky-big]

Operator add pulsar manager component success.

[nicknezis]

This is cool to see. A team I am on was going to start making a Pulsar Operator using Ansible OperatorSDK. https://github.com/operator-framework/operator-sdk/blob/master/doc/ansible/user-guide.md

Here is an Ansible Operator example for reference. https://github.com/tomaszkiewicz/mysql-operator/tree/335556ee3331ee7deba9049eafd67ad130125174/playbooks

Would the Pulsar team be interested in this work? Any opinions on Ansible vs Go?

[sijie]

I think both ansible and go operators can co-exist. I would also recommend submitting the operators to the operator hub as well so that it can be exposed to wider community.

[sky-big]

The next step is to publish to OperatorHub

[chrsoo]

@sijie yes, exactly. Should we take your comment regarding StreamNative to mean that you are planning to open source an Operator for Pulsar on Kubernetes that deals with the full scope of CRD's for Pulsar management? How soon is "soon"?

@sijie any feedback on this?

@sky-big regarding a specific design I would design it as explained above but I will see if I can produce a more detailed design specification as a PR.

@sky-big sorry for not getting back to you, the pulsar project took a back-seat for while where I work. It is getting reprioritised again so I will start looking into that PR again. Note that I am not talking about code at this point but at least I should be able to put how I think it should work.

[sky-big]

@chrsoo OK

[sky-big]

I created slack https://pulsaroperator.slack.com/

[nicknezis]

Why not just an #operator channel in the Pulsar Slack? We hang out in there so not sure if this is big enough effort to need a whole new Slack.

Having a design spec laid out is a great idea because we are starting to build something similar. If we have an agreed upon approach, members from my team could contribute functionality to flesh out the implementation.

Also we can perhaps keep separate Operators as Strimzi did for their Kafka Operators. This would allow the current big-sky Operator to exist for managing the cluster and my team could create another operator for managing the User,Topic, Tenant, Namespace type objects. https://strimzi.io/docs/overview/latest/#overview-components_str

[sky-big]

@nicknezis Hi, Sorry, I didn't know there was pulsar slack，now, I created one pulsar operator channel in the Pulsar Slack，We will discuss the following here.

There's another team looking at the same thing, we can discuss it together here. We've been talking about priorities for all function points, and we've been talking about rules.

[sky-big]

@nicknezis I‘ve invited you to join the channel

[blaghed]

Hi all, I'm very much interested in the discussion revolving around the Cluster and Resource Operators for Pulsar. So, kind of hope that the reason this ticket went quiet is that you moved the ongoing convo over to Slack? Is there any update from that side that would make sense adding here?

[tisonkun]

Open-ended discussion. Moved to the Discussions forum.