Allow kubernetes_logs source to function in a multi-tenanted k8s cluster without a ClusterRoleBinding

[chris1786]

Current Vector Version

vector 0.16.1

Use-cases

I have a scenario whereby I would like to use the kubernetes_logs source to collect the logs of the pods running within a single namespace on a multi-tenanted k8s cluster with rbac configured (such as OpenShift).

Attempted Solutions

It is currently possible to achieve the above by adding ClusterRoleBinding to a ClusterRole with watch permissions on namespaces and pods (as configured within the vault-agent helm chart). The extra_field_selector option can then be used to filter the pods by namespace.

Whilst this works, the requirement to have a ClusterRoleBinding and ClusterRole is not ideal.

Proposal

I would suggest adding an option which causes the watch api calls to be scoped to a specific set of namespaces. This would allow the ClusterRoleBinding and ClusterRole to be replaced with a RoleBinding and Role.

If this proposal is agreeable, I am happy to do the dev.

References

• 7589

[spencergilbert]

🤔 definitely not opposed.

Do you think this would look like a new namespace_selector option (or better name) in the source configuration?

Noting that we'd also need to update the k8s resources to handle this appropriately.

[chris1786]

🤔 definitely not opposed.

Do you think this would look like a new namespace_selector option (or better name) in the source configuration?

Noting that we'd also need to update the k8s resources to handle this appropriately.

This is essentially how I had envisaged it working, yes. I haven't yet worked out if the field should contain a single namespace, or a collection of namespaces.

Amending the k8s resources should be trivial. I note that there is both a helm chart, and a directory containing untemplated resource definitions. For the helm chart, I would envisage adding a value to toggle the switching-out of the ClusterRole and ClusterRoleBinding for a corresponding Role and RoleBinding. Would I need to make any changes to the untemplated resource definitions?

I plan on completing this as two change sets, the first to alter the application, and the second to amend the k8s resources

[spencergilbert]

👍 Is it worth replacing the ClusterRole with a Role or just the bindings? We should be able to use the current ClusterRole and bind per namespace with a RoleBinding?

I plan on completing this as two change sets, the first to alter the application, and the second to amend the k8s resources

That's perfect 👍

[gaetansnl]

Use case for this : each namespace has its own agent collecting logs from applications in the namespace with parsing rules, etc... And the global kubernetes agent running globally gets logs from the agents in the namespaces. It would allow us to decouple log configuration but still get all logs in the same place. Do you think there is a better way to achieve with the current version ?

[spencergilbert]

Use case for this : each namespace has its own agent collecting logs from applications in the namespace with parsing rules, etc... And the global kubernetes agent running globally gets logs from the agents in the namespaces. It would allow us to decouple log configuration but still get all logs in the same place. Do you think there is a better way to achieve with the current version ?

I don't believe so today @gaetansnl - it sounds like you'd need this issue for the per namespace agents, and then you could run Vector as an Aggregator to receive them from each agent.