Add a Kubernetes resource source

[joshgav]

Kubernetes provides a stream of resource create/update/delete events via its API as described here.

If Ansible could subscribe to this stream of events it could act as an out-of-band controller/operator. Similar concepts can be found in metacontroller and keptn.

To this end, could you please create a "source" based on this stream of Kubernetes resource-related events? Perhaps it could be configurable re which resource types it watches. Thank you!

---

NOTE: the stream of resource-related events is not the same as Kubernetes Events - capital E - which are a specific type of resource. Those would be nice to subscribe to as well but as another source; here's an article with more about them: https://www.cncf.io/blog/2021/12/21/extracting-value-from-the-kubernetes-events-feed/

[yajo]

If this were implemented, basically it would deprecate the whole Operator SDK for Ansible, which is way more alien for Ansible devs than EDA.

[joshgav]

This work from @sabre1041 seems to address this issue: https://www.ansible.com/blog/kubernetes-meets-event-driven-ansible

Thanks Andy! :partying_face:

[joshgav]

Hi @sabre1041, one thing I think I noted in reading your article is that we don't get all the properties of the Kubernetes resource itself in the rulebook. That is, we don't get the actual properties of the ConfigMap, Deployment or whatever CR was created. Those properties describe the desired state of the resource so they will often be needed by the reacting playbook.

The pattern in Go operators is almost always to start the reconciliation loop by retrieving the current resource from the API server. That is, after the operator is notified that a resource named x was ADDED/MODIFIED/DELETED it then goes and retrieves what exactly x currently is per the API server.

Perhaps in the future if users desire it we could enable this event source to make that callback automatically and "hydrate" all the resource properties in advance before handing it off to the playbook.

[ccamacho]

Hi @joshgav and @yajo I have been playing with EDA+Kubernetes and this might be interesting https://github.com/ansible/event-driven-ansible/pull/133

The main challenge here is how to enable the Python Kubernetes client without limiting the methods and parameters that can be used. There are still many things that needs to be done, like unit/integration/e2e testing, but if you find the pattern interesting I could definitely give it a push.

The main use case for this is that in some cases we might not want to handle everything with operators, with EDA we enable users to own their resources using an Ansible native workflow.

[the-it-jaeger]

I will echo my comment in the related PR:

I think there is a definite need for this. For example, I would like to have Ansible react when OCP ClusterAutoscaler adds or removes new Machines/Nodes by adding (or removing) their IP addresses to (or from) firewalls and load balancers.

I don't think an Ansible operator would be doing this kind of work.

[valkiriaaquatica]

I will echo my comment in the related PR:

I think there is a definite need for this. For example, I would like to have Ansible react when OCP ClusterAutoscaler adds or removes new Machines/Nodes by adding (or removing) their IP addresses to (or from) firewalls and load balancers.

I don't think an Ansible operator would be doing this kind of work.

I would like to share my experiences working with this Kubernetes plugin while deploying custom operators in clusters for a time.

When I deploy operators in a cluster (self made or not), they primarily handle internal configuration within the cluster, as we all know. But, with this Kubernetes plugin, I was able to control and automate tasks that extend beyond the cluster itself. Here are some examples where that i remember where I used it:

• Auto Scaling Deployments In auto scale deployments (basically AWS and Azure) when new nodes where added to a cluster, the "control" of this nodes, such as relevant metadta was send to ServiceNow to create this new object and also when the node was deleted, delete the object from ServiceNow
• External Load Balancers: When using outside loadbalancer which is not integrated in Kubernetes, EDA detects that a deployment consumes more amount of CPU (for example), EDA updates the rules in the external laodblancer like F5 or Nginx
• Hybrid Applications: In scenarios where applications were being migrated to Kubernetes or had parts deployed on mainframes and others on Kubernetes, EDA allowed me to detect changes in confiMaps or secrets.. in Kubernetes and quickly propagate those updates to the corresponding components on the mainframe :)

So honestly in my opinion it would be great to Red Hat tried to implement it on defaulf eda collection.

With all of these I had to say that with some methods of kubernetes the watch parameter of course is not necessary (idk like maybe VersionApi) so it could be change to optional and passed as an argument? @ccamacho I know it breaks the default behavior of the plugin btw

    extra_parameters = {
        *watch": True,  # could be optional
        "timeout_seconds": 10,
        "resource_version": last_resource_version,
    }