Lack of rate-limiting controls

[afrittoli]

Describe the bug

From the Tekton security audit:

Tekton Dashboard does not enforce rate limiting of HTTP requests. As a result, we were able to issue over a thousand requests in just over a minute.

Processing requests sent at such a high rate can consume an inordinate amount of resources, increasing the risk of denial-of-service attacks through excessive resource consumption. In particular, we were able to create hundreds of running “import resources” pods that were able to consume nearly all the host’s memory in the span of a minute.

Expected behaviour

Short term, implement rate limiting on all API endpoints.

Long term, run stress tests to ensure that the rate limiting enforced by Tekton Dashboard is robust.

Environment details

• Versions:
  • Tekton Dashboard: v0.24

[AlanGreene]

Given the Dashboard is not exposed outside the cluster by default, and that in full read-write mode should never be exposed publicly and should always be deployed behind a reverse proxy for authentication anyway, it could be argued that kubectl, tkn, and other clients are also affected by this issue.

For read-only mode there's less of a concern as new runs / related resources cannot be created in this mode, however large numbers of concurrent requests or a burst of requests in a short period to list / get / watch resources can indeed still cause high resource usage / slow responses.

I'm not sure if this is something that should be built in to the Dashboard application itself or would be better handled by a well-tested reverse proxy solution. Would it make sense to document and/or provide a simple copy-paste example showing how to achieve this with nginx + oauth2-proxy for example?

[tekton-robot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale with a justification. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close with a justification. If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[AlanGreene]

We should document an example for this, I'll try to put something together before end of year. /lifecycle frozen

[AlanGreene]

A few months ago I did an experiment to rewrite the Dashboard back-end entirely (as well as rearchitecting the client, some of which we've already recently adopted in https://github.com/tektoncd/dashboard/issues/2452 and related issues).

The resulting (partial) rewrite of the app, with the new back-end is in https://github.com/alangreene/dashboard-next

It includes rate-limiting support among other improvements. There will be some breaking changes in config so it'll have to be introduced in a non-breaking manner over a number of releases. I'll be creating issues in the next few weeks to track the various pieces of this with more details.