Different middlewares on different ports

Feature Request

Scope:

This proposal is about supporting the ability to specify different middlewares on different ports. It does not covers supporting multiple traffic-type on the same service.

Background:

Since the begining of Traefik Mesh, it has always been only possible to specify middlewares that impacts all ports defined within a service. However, these ports may have different constrains and different behavior. Right now, when a operator want different middleware for different ports he has to create multiple services with different middlewares.

Proposal:

Introduce a set of annotations that would allow overriding/setting configuration for a specific port. From an operator point of view, you could define either middlewares that are going to be applied on all ports or on one specific port. Annotations targetting a specific port would be used in favor of a potentially existing global value. A middleware option would be considered defined if its value is not zeroed.

First option: Backward compatible annotations

For this option, annotation would follow this form: mesh.traefik.io/ports.<port-name>.<middleware-option> The <port-name> represents the name given to the port in the Spec.Ports.Name field. The <middleware-option> is one of the following options:

retry-attempts
circuit-breaker-expression
ratelimit-average
ratelimit-burst

apiVersion: v1
kind: Service
metadata:
  name: test
  annotations:
    mesh.traefik.io/ratelimit-average: 12
    mesh.traefik.io/ratelimit-burst: 20

    mesh.traefik.io/ports.api.ratelimit-average: 100
    mesh.traefik.io/ports.api.ratelimit-burst: 200
    mesh.traefik.io/ports.api.retry-attempts: 4
    mesh.traefik.io/ports.api.circuit-breaker-expression: "LatencyAtQuantileMS(50.0) > 100"
spec:
  ports:
    - name: api
      port: 8080
      targetPort: 80
      protocol: TCP
    - name: web
      port: 8080
      targetPort: 80
      protocol: TCP
  selector:
    app: test

Second option: Restructured middleware structure

For this option, annotation would follow this form: mesh.traefik.io/ports.<port-name>.<middleware-name>[.<middleware-option>] The <port-name> represents the name given to the port in the Spec.Ports.Name field. The <middleware-name> represents the name of the middleware and <middleware-option> represent an option of the said middleware. Supported values are:

retry: with one option attempts.
circuitBreaker: with option expression.
ratelimit: with optionsaverageandburst`.

apiVersion: v1
kind: Service
metadata:
  name: test
  annotations:
    mesh.traefik.io/ratelimit.average: 12
    mesh.traefik.io/ratelimit.burst: 20

    mesh.traefik.io/ports.api.ratelimit.average: 100
    mesh.traefik.io/ports.api.ratelimit.burst: 200
    mesh.traefik.io/ports.api.retry.attempts: 4
    mesh.traefik.io/ports.api.circuitBreaker.expression: "LatencyAtQuantileMS(50.0) > 100"
spec:
  ports:
    - name: api
      port: 8080
      targetPort: 80
      protocol: TCP
    - name: web
      port: 8080
      targetPort: 80
      protocol: TCP
  selector:
    app: test

This option has the advantage of better structuring middlewares and given the ability to provide default values for these middlewares only if they are enabled.

traefik / mesh