[TRACKING] Discuss access to `falco` namespace

[incertum]

Discuss options available to grant us access to our falco namespace.

For instance this would be needed to retrieve Falco's own native metrics (currently piped to a log rotated file under /tmp/stats/), but more importantly so that we can check if everything is fine.

We are open to discuss various options to achieve these goals. The access model could also evolve over time as more projects onboard and as Falco as the first project has reached a stable overall deployment config.

CC @nikimanoledaki

[poiana]

There is not a label identifying the kind of this issue. Please specify it either using /kind <group> or manually from the side menu.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[nikimanoledaki]

We are creating a read/get/list-only Role / ServiceAccount / scoped kubeconfig as part of this issue: https://github.com/cncf-tags/green-reviews-tooling/issues/18

WDYT?

[incertum]

We could benefit from pods/exec resource and exec verb in order to kubectl cp the small files that contain our custom internal metrics. Open to any other approach.

@maxgio92 @leogr @LucaGuerra would you have other ideas?

[maxgio92]

Hi all, are those files required @incertum? I like the idea to keep minimum privileges @nikimanoledaki

[incertum]

Without having access to the internal metrics we write to files, I am not sure how to check on the kernel event rates and other metrics. Perhaps down the road we can expose them in a better way, such as over Prometheus, but Falco does not have a Prometheus exporter quite yet.

One possibility could be to grant us such access until the CNCF testbed is more established and we have implemented a metrics Prometheus exporter? Or we defer inspecting the internal Falco metrics until we have everything in place -- equally valid.

[rossf7]

Hi @incertum, ideally we would query the metrics from Prometheus but until there is an exporter available we can get you access to these log files.

I'll create a separate kubeconfig and service account that also has the pods/exec resource and exec verb based on the readonly kubeconfig we already have.

https://github.com/cncf-tags/green-reviews-tooling/blob/main/docs/read-only-kubeconfig.md

and we'll provide access via 1Password once we have our account setup.

https://github.com/cncf-tags/green-reviews-tooling/issues/37

cc @nikimanoledaki @AntonioDiTuri

[AntonioDiTuri]

Maybe to while waiting for the 1Password account we could share privately the new kubeconfig so that Melissa is able to check. What do you think?

[incertum]

Thank you @AntonioDiTuri confirming that I have received the interim kubeconfig and it is all working.

[incertum]

I think we can mark this issue as completed. Once we have all metrics exposed over Prometheus we will revoke my interim admin access and reduce the access scope as discussed above. We all agreed on the ideal end state.