Extract Kubernetes source feature as a plugin

huseyinbabal commented 2 years ago

Importan Update

Kubernetes plugin is successfully extracted as a plugin, and it works now as expected, but since entire Botkube system depends on Kubernetes event, we needed to touch all kind of component including filters, recommendations, reporting, etc... We introduced follow-up items as follows;

[x] #993 - @mszostok
[x] #994 - @mszostok
[x] #995 - @mszostok
[x] #996 - @mszostok
[x] #997 - @mszostok
[x] #998
[x] #999
[x] #1000 - @mszostok
[x] Remove botkube.io/channel: {channel} annotation from k8s source

Out of scope:

[x] #1009
[x] #1001

Overview

Botkube supports Kubernetes events. It is a typical example of a Source plugin. We need to extract it as a standalone plugin.

Acceptance Criteria

Once we extract this as a plugin, it will be a sub-process in the pod which means it will use the same service account as other plugins will. For now, we can assume that customer will prepare a unified role and attach it to the service account. This is something like using your workstation machine for kubectl, helm, and does not care which service account you use in the beginning.
Currently, Kubernetes source events are tightly coupled with Botkube, we can add a sourcing mechanism as a Kubernetes plugin. You can see an independent example here
It should be background compatible.
Plugin-specific parameters should be passed during plugin initialization. Hashicorp's Go plugin system runs the plugin executable as a subprocess, so it also accepts exec arguments where we can resolve configuration and pass them as separate arguments
Plugin parameters will be documented in the plugin's README.md page, and they can be provided from Botkube as an environment variable like KUBERNETES_... so that they can be passed to the executable in this line
Since each source plugin has different data structure, we cannot have a general data structure except we specified #Cloud Events. data field has free data structure, but we can convert them to map then extract any section based on customer definition by using Go template. data field has free flat structure, a simple key-value pairs with one level, but we can put mandatory fields in this struct like message, user, etc. Or we can put mandatory fields to cloud events schema as shown below.
```
{
"id" : "e2361318-e50b-472e-8b17-13dbac1daac1",   
"source" : "Kubernetes",     
"specversion" : "1.0",                             
"time" : "2022-03-23T14:35:39.738Z",               
"message": "DB instance not found",
"user": "john",
"data" : {                                         
"namespace" : "Botkube",
"deploymentName" : "test",
"replicas" : 5
}
}
```
Update gRPC source API
Make sure anonymized telemetry is still supported - we can lose some data, but confirm with @brampling if we can do that
Update the document

Additional scope / notes

Ensure it supports the RBAC design according to #712 (design https://github.com/kubeshop/botkube/blob/main/docs/proposal/2022-12-23-rbac.md), if #934 is already done. If not, ensure we can easily support this in the plugin.

mszostok commented 1 year ago

NOTE: as a part of this task we should also move the filter logic into Source Dispatcher service: https://github.com/kubeshop/botkube/blob/main/internal/source/dispatcher.go#L33

mszostok commented 1 year ago

After sync with @huseyinbabal and @pkosiec we decided to remove support for the Botkube related annotations as described in https://github.com/kubeshop/botkube/issues/690

Instead, users will be able to define annotation constrains directly in k8s source configuration and bind it per channel.

AC

update annotation and labels constrains on k8s source to allow setting up include and exclude.

kubeshop / botkube