eguzki commented 9 months ago

What

Kuadrant extensions for the OpenAPI Specification (OAS) 3.x to generate:

[X] The command kuadrantctl generate gatewayapi httproute to create Gateway API HTTPRoute from OpenAPI Specification (OAS) 3.x powered with kuadrant extensions
[X] The kuadrantctl generate kuadrant ratelimitpolicy command generates an Kuadrant RateLimitPolicy from your OpenAPI Specification (OAS) 3.x powered with kuadrant extensions.
[X] The command kuadrantctl generate kuadrant authpolicy to create kuadrant Auth Policy from OpenAPI Specification (OAS) 3.x powered with kuadrant extensions #46 #50
[X] #48
[X] #49

Info level kuadrant extension

Kuadrant extension that can be added at the info level of the OpenAPI spec.

info:
  x-kuadrant:
    route:  ## HTTPRoute metadata
      name: "petstore"
      namespace: "petstore"
      labels:  ## map[string]string
        deployment: petstore
      hostnames:  ## []gateway.networking.k8s.io/v1beta1.Hostname
        - example.com
      parentRefs:  ## []gateway.networking.k8s.io/v1beta1.ParentReference
        - name: apiGateway
          namespace: gateways

Path level kuadrant extension

Kuadrant extension that can be added at the path level of the OpenAPI spec. This configuration at the path level is the default when there is no operation level configuration.

paths:
  /cat:
    x-kuadrant:  ## Path level Kuadrant Extension
      disable: true  ## Remove from the HTTPRoute. Optional. Default: false
      pathMatchType: Exact ## Specifies how to match against the path Value. Valid values: [Exact;PathPrefix]. Optional. Default: Exact
      backendRefs:  ## Backend references to be included in the HTTPRoute. []gateway.networking.k8s.io/v1beta1.HTTPBackendRef. Optional.
        - name: petstore
          port: 80
          namespace: petstore
      rate_limit:  ## Rate limit config. Optional.
        rates:   ## Kuadrant API []github.com/kuadrant/kuadrant-operator/api/v1beta2.Rate
          - limit: 1
            duration: 10
            unit: second
        counters:   ## Kuadrant API []github.com/kuadrant/kuadrant-operator/api/v1beta2.CountextSelector
          - auth.identity.username
        when:   ## Kuadrant API []github.com/kuadrant/kuadrant-operator/api/v1beta2.WhenCondition
          - selector: metadata.filter_metadata.envoy\.filters\.http\.ext_authz.identity.userid
            operator: eq
            value: alice

Operation level kuadrant extension

Kuadrant extension that can be added at the operation level of the OpenAPI spec. Same schema as path level kuadrant extension.

paths:
  /cat:
    get:
      x-kuadrant:  ## Path level Kuadrant Extension
        disable: true  ## Remove from the HTTPRoute. Optional. Default: path level "disable" value
        pathMatchType: Exact ## Specifies how to match against the path Value. Valid values: [Exact;PathPrefix]. Optional. Default: Exact
        backendRefs:  ## Backend references to be included in the HTTPRoute. Optional.
          - name: petstore
            port: 80
            namespace: petstore
        rate_limit:  ## Rate limit config. Optional.
          rates:   ## Kuadrant API github.com/kuadrant/kuadrant-operator/api/v1beta2.Rate
            - limit: 1
              duration: 10
              unit: second
          counters:   ## Kuadrant API github.com/kuadrant/kuadrant-operator/api/v1beta2.CountextSelector
            - auth.identity.username
          when:   ## Kuadrant API github.com/kuadrant/kuadrant-operator/api/v1beta2.WhenCondition
            - selector: metadata.filter_metadata.envoy\.filters\.http\.ext_authz.identity.userid
              operator: eq
              value: alice

Verification Steps

Clone the repo and checkout to the PR branch httproute-kuadrant-extensions

git clone https://github.com/Kuadrant/kuadrantctl.git
cd kuadrantctl
git checkout httproute-kuadrant-extensions

Setup cluster, istio and Gateway API CRDs
```
make local-setup
```
Build and install CLI in bin/kuadrantctl path
```
make install
```
Install Kuadrant service protection. The CLI can be used to install kuadrant v0.4.1
```
bin/kuadrantctl install
```

Deploy petstore backend API

kubectl create namespace petstore
kubectl apply -n petstore -f examples/petstore/petstore.yaml

Let's create Petstore's OpenAPI spec

```yaml cat <petstore-openapi.yaml --- openapi: "3.0.3" info: title: "Pet Store API" version: "1.0.0" x-kuadrant: route: name: "petstore" namespace: "petstore" hostnames: - example.com parentRefs: - name: istio-ingressgateway namespace: istio-system servers: - url: https://example.io/v1 paths: /cat: x-kuadrant: ## Path level Kuadrant Extension backendRefs: - name: petstore port: 80 namespace: petstore rate_limit: rates: - limit: 1 duration: 10 unit: second counters: - request.headers.x-forwarded-for get: # Added to the route and rate limited operationId: "getCat" responses: 405: description: "invalid input" post: # NOT added to the route x-kuadrant: ## Operation level Kuadrant Extension disable: true operationId: "postCat" responses: 405: description: "invalid input" /dog: get: # Added to the route and rate limited x-kuadrant: ## Operation level Kuadrant Extension backendRefs: - name: petstore port: 80 namespace: petstore rate_limit: rates: - limit: 3 duration: 10 unit: second counters: - request.headers.x-forwarded-for operationId: "getDog" responses: 405: description: "invalid input" post: # Added to the route and NOT rate limited x-kuadrant: ## Operation level Kuadrant Extension backendRefs: - name: petstore port: 80 namespace: petstore operationId: "postDog" responses: 405: description: "invalid input" /mouse: get: # NOT added to the route x-kuadrant: ## Operation level Kuadrant Extension disable: true operationId: "getMouse" responses: 405: description: "invalid input" EOF ```

NOTE: servers base path not included. WIP in following up PRs.

Operation	Applied config
`GET /cat`	It should return 200 Ok and be rate limited (1 req / 10 seconds)
`POST /cat`	Not added to the HTTPRoute. It should return 404 Not Found
`GET /dog`	It should return 200 Ok and be rate limited (3 req / 10 seconds)
`POST /dog`	It should return 200 Ok and NOT rate limited
`GET /mouse`	Not added to the HTTPRoute. It should return 404 Not Found

Create the HTTPRoute using the CLI

bin/kuadrantctl generate gatewayapi httproute --oas petstore-openapi.yaml | kubectl apply -n petstore -f -

Create the Rate Limit Policy

bin/kuadrantctl generate kuadrant ratelimitpolicy --oas petstore-openapi.yaml | kubectl apply -n petstore -f -

Test OpenAPI endpoints
- GET /cat -> It should return 200 Ok and be rate limited (1 req / 10 seconds)

curl --resolve example.com:9080:127.0.0.1 -v "http://example.com:9080/cat"

POST /cat -> Not added to the HTTPRoute. It should return 404 Not Found

curl --resolve example.com:9080:127.0.0.1 -v -X POST "http://example.com:9080/cat"

GET /dog -> It should return 200 Ok and be rate limited (3 req / 10 seconds)

curl --resolve example.com:9080:127.0.0.1 -v "http://example.com:9080/dog"

POST /dog -> It should return 200 Ok and NOT rate limited

curl --resolve example.com:9080:127.0.0.1 -v -X POST "http://example.com:9080/dog"

GET /mouse -> Not added to the HTTPRoute. It should return 404 Not Found

curl --resolve example.com:9080:127.0.0.1 -v -X POST "http://example.com:9080/mouse"

Clean environment
```
make local-cleanup
```

codecov-commenter commented 9 months ago

Codecov Report

Attention: 353 lines in your changes are missing coverage. Please review.

Comparison is base (12b3aa3) 0.55% compared to head (6405744) 0.38%.

Files	Patch %	Lines
pkg/utils/oas_utils.go	0.00%	130 Missing :warning:
cmd/generate_kuadrant_authpolicy.go	0.00%	65 Missing :warning:
cmd/generate_kuadrant_ratelimitpolicy.go	0.00%	61 Missing :warning:
pkg/utils/kuadrant_oas_extension_types.go	0.00%	55 Missing :warning:
cmd/generate_gatewayapi_httproute.go	0.00%	13 Missing :warning:
cmd/install.go	0.00%	11 Missing :warning:
pkg/utils/maps.go	0.00%	9 Missing :warning:
cmd/generate_kuadrant.go	0.00%	3 Missing :warning:
pkg/utils/external_resource_reader.go	0.00%	3 Missing :warning:
pkg/utils/k8s_utils.go	0.00%	2 Missing :warning:
... and 1 more

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #43 +/- ## ======================================== - Coverage 0.55% 0.38% -0.18% ======================================== Files 13 17 +4 Lines 538 783 +245 ======================================== Hits 3 3 - Misses 535 780 +245 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

thomasmaas commented 9 months ago

Nice to see ideas around this. Wondering if there's a set of Gateway API related instructions that would make sense outside of Kuadrant as well, and thus maybe should live in a separate x-context?

jasonmadigan commented 9 months ago

Nice one @eguzki!

Giving it a try.

kuadrantctl generate gatewayapi httproute --oas petstore-openapi.yaml
Error: required flag(s) "gateway", "namespace", "public-host", "service-name" not set

Maybe I've missed something...

---
openapi: "3.0.3"
info:
  title: "Pet Store API"
  version: "1.0.0"
  x-kuadrant:
    route:
      name: "petstore"
      namespace: "petstore"
      hostnames:
        - example.com
      parentRefs:
        - name: istio-ingressgateway
          namespace: istio-system
servers:
  - url: https://example.io/v1
paths:
  /cat:
    x-kuadrant:  ## Path level Kuadrant Extension
      enable: true
      backendRefs:
        - name: petstore
          port: 80
          namespace: petstore
      rate_limit:
        rates:
          - limit: 1
            duration: 10
            unit: second
        counters:
          - request.headers.x-forwarded-for
    get:  # Added to the route and rate limited
      operationId: "getCat"
      responses:
        405:
          description: "invalid input"
    post:  # NOT added to the route
      x-kuadrant:  ## Operation level Kuadrant Extension
        enable: false
        backendRefs:
          - name: petstore
            port: 80
            namespace: petstore
        rate_limit:
          rates:
            - limit: 2
              duration: 10
              unit: second
          counters:
            - request.headers.x-forwarded-for
      operationId: "postCat"
      responses:
        405:
          description: "invalid input"
  /dog:
    get:  # Added to the route and rate limited
      x-kuadrant:  ## Operation level Kuadrant Extension
        enable: true
        backendRefs:
          - name: petstore
            port: 80
            namespace: petstore
        rate_limit:
          rates:
            - limit: 3
              duration: 10
              unit: second
          counters:
            - request.headers.x-forwarded-for
      operationId: "getDog"
      responses:
        405:
          description: "invalid input"
    post:  # Added to the route and NOT rate limited
      x-kuadrant:  ## Operation level Kuadrant Extension
        enable: true
        backendRefs:
          - name: petstore
            port: 80
            namespace: petstore
      operationId: "postDog"
      responses:
        405:
          description: "invalid input"
  /mouse:
    get:  # NOT added to the route
      operationId: "getMouse"
      responses:
        405:
          description: "invalid input"

Wondered if we could also look at a sample script to use this against api-poc-petstore?