3forges / poc-k8s-keycloak-auth-sidecar

A poc to add OpenID Connect auth to a Rest API, using Keycloak and the SideCar pattern
0 stars 0 forks source link

Topic #1

Open Jean-Baptiste-Lasselle opened 11 months ago

Jean-Baptiste-Lasselle commented 11 months ago

I want to add OpenID COnnect auth to my rest api, using auth sidecar (scalable authentication)

I want to find out if it is possible with bitnami's keycloak's helm chart, to set up a keycloak gatekeeper side car :

I want to experiment more Kubernetes Admission controllers: Portieris and Open Policy Agent Gatekeeper

OPA Gatekeeper :

Portieris :

Jean-Baptiste-Lasselle commented 11 months ago
package istio.authz

import input.attributes.request.http as http_request

default allow = false

allow = response {
  http_request.method == "GET"
  response := {
      "allowed": true,
      "headers": {"X-Auth-User": "1234"}
  }
}

In our policy.regofile, we set istio/authz/allow to response (where "allowed": True ) should all statements within that rule validates true (i.e. if http_request.method == "GET" is true).

Autre élément pour comprendre comment l'agent opa est configuré avec les fichiers rego :

image

Jean-Baptiste-Lasselle commented 11 months ago
export PESTO_TOKEN="SFMyNTY.g3QAAAAGbQAAAAtfY3NyZl90b2tlbm0AAAAYZ3RTay1EbGFtTE51cnkzcFlQd3gydjFkbQAAAA9jdXJyZW50X3VzZXJfaWRhAW0AAAAJbGFzdF9zZWVuYmVCM_ttAAAAJXBsYXVzaWJsZS5wZXN0by5pb19vZmZlcl9lbWFpbF9yZXBvcnRkAAR0cnVlbQAAABJzZXNzaW9uX3RpbWVvdXRfYXRiZVSrS20AAAAodGVzdC13ZWJzaXRlLnBva3VzLmlvX29mZmVyX2VtYWlsX3JlcG9ydGQABHRydWU.bkr_zPZ51MLOp-PZhwVEyN4ivpWE1d91TT5hgKCRzTA"

curl -X GET \
       -H "Accept: application/json;pesto" \
       -H "Content-Type: application/json;pesto" \
       -H "Authorization: Bearer ${PESTO_TOKEN}" \
       -H GET \
       http://plausible.pesto.io:8080/anything | jq .
curl -X GET        -H "Accept: application/json;pesto"        -H "Content-Type: application/json;pesto"        -H "X-Auth-User ${PESTO_TOKEN}"        -H GET        http://plausible.pesto.io:8080/anything | jq .

Ok il y a un *.rego possible pour intégrer à keycloak:

package oidc

issuers := {"https://issuer1.example.com", "https://issuer2.example.com"}

metadata_discovery(issuer) := http.send({
    "url": concat("", [issuers[issuer], "/.well-known/openid-configuration"]),
    "method": "GET",
    "force_cache": true,
    "force_cache_duration_seconds": 86400 # Cache response for 24 hours
}).body

claims := jwt.decode(input.token)[1]
metadata := metadata_discovery(claims.iss)

jwks_endpoint := metadata.jwks_uri
token_endpoint := metadata.token_endpoint
rload.global_downstream_max_connections
envoy_1  | [2023-11-01 18:05:26.749][8][info][main] [source/server/drain_manager_impl.cc:70] shutting down parent after drain
app_1    | [2023-11-01 17:50:14 +0000] [1] [INFO] Starting gunicorn 19.9.0
app_1    | [2023-11-01 17:50:14 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
app_1    | [2023-11-01 17:50:14 +0000] [1] [INFO] Using worker: gevent
app_1    | [2023-11-01 17:50:14 +0000] [10] [INFO] Booting worker with pid: 10
opa_1    | error: load error: 1 error occurred during loading: /config/policy.rego:5: rego_parse_error: functions must use = operator (not := operator)
opa_1    |      metadata_discovery(issuer) := http.send({
opa_1    |                                 ^
envoy-opa-compose_opa_1 exited with code 1
package oidc

issuers := {"https://issuer1.example.com", "https://issuer2.example.com"}

metadata_discovery(issuer) = http.send({
    "url": concat("", [issuers[issuer], "/.well-known/openid-configuration"]),
    "method": "GET",
    "force_cache": true,
    "force_cache_duration_seconds": 86400 # Cache response for 24 hours
}).body

claims = jwt.decode(input.token)[1]
metadata = metadata_discovery(claims.iss)

jwks_endpoint = metadata.jwks_uri
token_endpoint = metadata.token_endpoint
Jean-Baptiste-Lasselle commented 11 months ago
version: "3.7"
services:
  envoy:
    build: ./compose/envoy
    ports:
      - "8080:80"
    volumes:
      - ./envoy.yaml:/config/envoy.yaml
    environment:
      - DEBUG_LEVEL=info
      - SERVICE_NAME=app  # should match name of upstream service
      - SERVICE_PORT=80

  opa:
    image: openpolicyagent/opa:0.26.0-envoy
    volumes:
      - ./policy.rego:/config/policy.rego
    command:
      - "run"
      - "--log-level=debug"
      - "--log-format=json-pretty"
      - "--server"
      - "--set=plugins.envoy_ext_authz_grpc.path=envoy/authz/allow"  # default value
      - "--set=decision_logs.console=true"
      - "/config/policy.rego"

  app:
    image: kennethreitz/httpbin:latest
Jean-Baptiste-Lasselle commented 11 months ago
export KC_SERVER_URL=http://localhost:8081/auth
export KC_ADMIN_USER=admin
export KC_ADMIN_PASSWORD=admin
export KC_REALM=demo

docker run \
  -d \
  --rm \
  --name keycloak-opa \
  -e KEYCLOAK_ADMIN=$KC_ADMIN_USER \
  -e KEYCLOAK_ADMIN_PASSWORD=$KC_ADMIN_PASSWORD \
  -e KC_HTTP_RELATIVE_PATH=auth \
  -p 8081:8080 \
  quay.io/keycloak/keycloak:19.0.1 start-dev
version: "3.7"
services:
  envoy:
    build: ./compose/envoy
    ports:
      - "8080:80"
    volumes:
      - ./envoy.yaml:/config/envoy.yaml
    environment:
      # - DEBUG_LEVEL=info
      - DEBUG_LEVEL=debug
      - SERVICE_NAME=app  # should match name of upstream service
      - SERVICE_PORT=80
    extra_hosts:
        - "envoy.pesto.io:192.168.181.202"
        - "keycloak.pesto.io:192.168.181.202"
        - "opa.pesto.io:192.168.181.202"
        - "app.pesto.io:192.168.181.202"
        - "testwebsite.pokus.io:192.168.181.236"
    networks:
      pesto_net:
        aliases:
          - envoy.pesto.io

  keycloak:
    image: "quay.io/keycloak/keycloak:19.0.1"
    command: 'start-dev'
    ports:
      - "8081:8080"
    # volumes:
    #  - ./envoy.yaml:/config/envoy.yaml
    environment:
      - KC_SERVER_URL=http://keycloak.pesto.io:8081/auth
      # - KC_SERVER_URL=http://localhost:8080/auth
      - KC_ADMIN_USER=admin
      - KC_ADMIN_PASSWORD=admin
      - KC_REALM=pesto
      - KEYCLOAK_ADMIN=pesto
      - KEYCLOAK_ADMIN_PASSWORD=pesto
      - KC_HTTP_RELATIVE_PATH=auth
      # - KC_DB=postgres
      # - KC_DB_URL=<DBURL>
      # - KC_DB_USERNAME=<DBUSERNAME>
      # - KC_DB_PASSWORD=<DBPASSWORD>
      - KC_HOSTNAME=keycloak.pesto.io
    extra_hosts:
        - "keycloak.pesto.io:192.168.181.202"
        - "testwebsite.pokus.io:192.168.181.236"
    networks:
      pesto_net:
        aliases:
          - keycloak.pesto.io

  opa:
    image: openpolicyagent/opa:0.26.0-envoy
    volumes:
      - ./policy.rego:/config/policy.rego
    command:
      - "run"
      - "--log-level=debug"
      - "--log-format=json-pretty"
      - "--server"
      - "--set=plugins.envoy_ext_authz_grpc.path=envoy/authz/allow"  # default value
      - "--set=decision_logs.console=true"
      - "/config/policy.rego"
    extra_hosts:
        - "envoy.pesto.io:192.168.181.202"
        - "keycloak.pesto.io:192.168.181.202"
        - "opa.pesto.io:192.168.181.202"
        - "app.pesto.io:192.168.181.202"
        - "testwebsite.pokus.io:192.168.181.236"
    networks:
      pesto_net:
        aliases:
          - opa.pesto.io
  app:
    image: kennethreitz/httpbin:latest
    extra_hosts:
        - "envoy.pesto.io:192.168.181.202"
        - "keycloak.pesto.io:192.168.181.202"
        - "opa.pesto.io:192.168.181.202"
        - "app.pesto.io:192.168.181.202"
        - "testwebsite.pokus.io:192.168.181.236"
    networks:
      pesto_net:
        aliases:
          - app.pesto.io

# --- --- ---  
#  + 
networks:
  pesto_net:
  pokus_net:
package oidc

# issuers := {"https://issuer1.example.com", "https://issuer2.example.com"}

# http://keycloak.pesto.io:8080/auth
issuers := {"http://keycloak.pesto.io:8081/auth", "http://keycloak.pesto.io:8081/auth/realms/pesto", "http://keycloak.pesto.io:8081/auth/realms/pesto/.well-known/openid-configuration"}

metadata_discovery(issuer) = http.send({
    "url": concat("", [issuers[issuer], "/.well-known/openid-configuration"]),
    "method": "GET",
    "force_cache": true,
    "force_cache_duration_seconds": 86400 # Cache response for 24 hours
}).body

claims = jwt.decode(input.token)[1]
metadata = metadata_discovery(claims.iss)

jwks_endpoint = metadata.jwks_uri
token_endpoint = metadata.token_endpoint
Jean-Baptiste-Lasselle commented 11 months ago

see also: https://dev.to/gabrielbiasi/automatic-sso-in-kubernetes-workloads-using-a-sidecar-container-3752

Jean-Baptiste-Lasselle commented 11 months ago

celui-là je dois absolument le faire : https://dev.to/techworld_with_nana/how-to-setup-a-keycloak-gatekeeper-to-secure-the-services-in-your-kubernetes-cluster-5d2d

Jean-Baptiste-Lasselle commented 11 months ago

ok, maintenant j'ai en plus un fichier json de définition du realm pesto :

podman|docker run --name keycloak_unoptimized -p 8080:8080 \
        -e KEYCLOAK_ADMIN=admin -e KEYCLOAK_ADMIN_PASSWORD=change_me \
        -v /path/to/realm/data:/opt/keycloak/data/import \
        quay.io/keycloak/keycloak:latest \
        start-dev --import-realm

image

Jean-Baptiste-Lasselle commented 11 months ago

ok donc là j'ai mon keycloak en mode dev, et j'ai mon authorization endpoint pour le "login with keycloak" :

curl -iv http://keycloak.pesto.io:8081/auth/realms/pesto/.well-known/openid-configuration | tail -n 1 | jq .
Jean-Baptiste-Lasselle commented 11 months ago

ok et là c'est gagné :

image

and in keycloak i see the logged in user session:

image

i can even force logging him out :

image

et voilà les 2 reqêtes OpenID qui suivent pour la redirection :

image

image

image

Jean-Baptiste-Lasselle commented 11 months ago

le flow marche aussi pour le type token au lieu de code, mais il a fallu cocher le Implicit Flow, sinon ça ne marchait pas :

image

http://testwebsite.pokus.io:5173/projects#session_state=ead95133-eb93-4889-8507-51ea73e3cb51&access_token=eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJCRk0tOURXbzctd09Pb3ladFlHRUptZVdpWGZJOEhyN2hJbm1seFBwNG1NIn0.eyJleHAiOjE2OTg4NzcwODYsImlhdCI6MTY5ODg3NjE4NiwiYXV0aF90aW1lIjoxNjk4ODc2MTg2LCJqdGkiOiJmMDRlYjJiYi0zZGI0LTQ5ZGYtYWEwNy0zNDdmZDAwNmY3MWUiLCJpc3MiOiJodHRwOi8va2V5Y2xvYWsucGVzdG8uaW86ODA4MS9hdXRoL3JlYWxtcy9wZXN0byIsImF1ZCI6ImFjY291bnQiLCJzdWIiOiI4MDkzZDY3Yy05ZGE0LTQzMWEtOGIzMS1iMzYwNzkzNWFlMjAiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJwZXN0by1vaWRjLWNsaWVudC1pZCIsInNlc3Npb25fc3RhdGUiOiJlYWQ5NTEzMy1lYjkzLTQ4ODktODUwNy01MWVhNzNlM2NiNTEiLCJhY3IiOiIxIiwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbIm9mZmxpbmVfYWNjZXNzIiwiZGVmYXVsdC1yb2xlcy1wZXN0byIsInVtYV9hdXRob3JpemF0aW9uIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5hZ2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJlbWFpbCBwb2t1cyBwcm9maWxlIHBlc3RvLXJlYWRlciIsInNpZCI6ImVhZDk1MTMzLWViOTMtNDg4OS04NTA3LTUxZWE3M2UzY2I1MSIsImVtYWlsX3ZlcmlmaWVkIjpmYWxzZSwibmFtZSI6IkplYW4tQmFwdGlzdGUgTGFzc2VsbGUiLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJwZXN0byIsImdpdmVuX25hbWUiOiJKZWFuLUJhcHRpc3RlIiwiZmFtaWx5X25hbWUiOiJMYXNzZWxsZSIsImVtYWlsIjoiamVhbi5iYXB0aXN0ZS5sYXNzZWxsZUBnbWFpbC5jb20ifQ.CR_LAMJzWHQbSs1hV1apCleoWdSRtjbCpiRPtR3EPGEt12TpdkfYMHzpJ3Yx4E_HKjEdaG33pCLw5m5mnWxKL8Ytux9Xq-trM_7lxmvS8ZXs45xlEhpjAI0t9ASRSed29PaJpgVK6y3zhSzNv-XdSLIFWgekBp14hGfJlHTZAJ8SblS7B1dBHeSDq7fNTceqkxPfy3y46_T4K_ykoC2L98ByicXPrXb1792YchWGVpSfzn2G1b8zbP72DLLJ2orG97kr2KpdwiEJLVSPSojzkQPJ2HKMqiRnGkIA_r8FS5-0Jq5LVAPR8l8h4yF9Kn6i4O-a4y9Yz2Lx8hw8MFObTg&token_type=Bearer&expires_in=900

de plus, j'ai eut le "do you confirm you grant the following permissions ? ) , grâce à ces 2 options de configurations du client OIDC :

image

image

image

Jean-Baptiste-Lasselle commented 11 months ago

ah pour pouvoir faire un revoke des granted scope pour l'API il suffit de faire un impersonate dans la gestion des users, et derrière aller à la gestyion des applications

image

image

Jean-Baptiste-Lasselle commented 11 months ago

J'ai pas fini, je ne sais pas comment choper un token et autntifier auprès de l'api httpbin

bref je veux verifier l'intégration keycloak ds le fichier *.rego

Jean-Baptiste-Lasselle commented 11 months ago

et enfin il faudra ajouter un adapter /client nestjs pour la REST API

Jean-Baptiste-Lasselle commented 11 months ago

ps:

https://docs.sendgrid.com/api-reference/api-key-permissions/api-key-permissions#admin-api-key-permissions

Jean-Baptiste-Lasselle commented 11 months ago

Login with Github

export GITHUB_AUTHORIZATION_CODE='xxxxxxxxxxxx'
export GH_CLIENT_ID='xxxxxxxxxxx'
export GH_CLIENT_SECRET='xxxxxxxxxxxxx'
export GH_OAUTH_REDIRECT_URI='http%3A%2F%2Ftestwebsite.pokus.io%3A5173%2Foauth%2Fgithub%2Fcallback'

export QURY_PARAMS="redirect_uri=${GH_OAUTH_REDIRECT_URI}&code=${GITHUB_AUTHORIZATION_CODE}&client_id=${GH_CLIENT_ID}&client_secret=${GH_CLIENT_SECRET}"
curl -X POST \
       -H 'Content-Type: application/json' \
       -H 'Accept: application/json' \
       https://github.com/login/oauth/access_token?${QURY_PARAMS} | tail -n1 | jq .
export GH_TOKEN='your token here'

curl -iv -X GET \
            -H "Authorization: Bearer ${GH_TOKEN}" \
            -H "Content-Type: application/json" \
            https://api.github.com/user

There might be a good example to try for the GIthub OAuth : https://redux-toolkit.js.org/rtk-query/usage/automated-refetching#providing-errors-to-the-cache

Jean-Baptiste-Lasselle commented 11 months ago

metadata_discovery(issuer) = http.send({

oh, about my rego file i found something in my setup :

WARNING: Some networks were defined but are not used by any service: pokus_net
Attaching to envoy-opa-compose_opa_1
opa_1       | error: initialization error: 1 error occurred: /config/policy.rego:15: rego_type_error: undefined function jwt.decode
opa_1       | error: initialization error: 1 error occurred: /config/policy.rego:15: rego_type_error: undefined function jwt.decode
envoy-opa-compose_opa_1 exited with code 1
jbl@pokus2:~/envoy-opa-compose$
package oidc

# Get the JWT value from the query `input`
jwt := input.jwt

# issuers := {"https://issuer1.example.com", "https://issuer2.example.com"}

# http://keycloak.pesto.io:8080/auth
issuers := {"http://keycloak.pesto.io:8081/auth", "http://keycloak.pesto.io:8081/auth/realms/pesto", "http://keycloak.pesto.io:8081/auth/realms/pesto/.well-known/openid-configuration"}

metadata_discovery(issuer) = http.send({
    "url": concat("", [issuers[issuer], "/.well-known/openid-configuration"]),
    "method": "GET",
    "force_cache": true,
    "force_cache_duration_seconds": 86400 # Cache response for 24 hours
}).body

claims = jwt.decode(input.token)[1]
metadata = metadata_discovery(claims.iss)

jwks_endpoint = metadata.jwks_uri
token_endpoint = metadata.token_endpoint
$ docker-compose logs -f opa
WARNING: Some networks were defined but are not used by any service: pokus_net
Attaching to envoy-opa-compose_opa_1
opa_1       | error: initialization error: 1 error occurred: /config/policy.rego:18: rego_type_error: undefined function data.oidc.jwt.decode
envoy-opa-compose_opa_1 exited with code 1
package oidc

# Get the JWT value from the query `input`
# jwt := input.jwt

# issuers := {"https://issuer1.example.com", "https://issuer2.example.com"}

# http://keycloak.pesto.io:8080/auth
issuers := {"http://keycloak.pesto.io:8081/auth", "http://keycloak.pesto.io:8081/auth/realms/pesto", "http://keycloak.pesto.io:8081/auth/realms/pesto/.well-known/openid-configuration"}

metadata_discovery(issuer) = http.send({
    "url": concat("", [issuers[issuer], "/.well-known/openid-configuration"]),
    "method": "GET",
    "force_cache": true,
    "force_cache_duration_seconds": 86400 # Cache response for 24 hours
}).body

claims = io.jwt.decode(input.token)[1]
metadata = metadata_discovery(claims.iss)

jwks_endpoint = metadata.jwks_uri
token_endpoint = metadata.token_endpoint

Alright, now, when i send this request :

export PESTO_TOKEN="SFMyNTY.g3QAAAAGbQAAAAtfY3NyZl90b2tlbm0AAAAYZ3RTay1EbGFtTE51cnkzcFlQd3gydjFkbQAAAA9jdXJyZW50X3VzZXJfaWRhAW0AAAAJbGFzdF9zZWVuYmVCM_ttAAAAJXBsYXVzaWJsZS5wZXN0by5pb19vZmZlcl9lbWFpbF9yZXBvcnRkAAR0cnVlbQAAABJzZXNzaW9uX3RpbWVvdXRfYXRiZVSrS20AAAAodGVzdC13ZWJzaXRlLnBva3VzLmlvX29mZmVyX2VtYWlsX3JlcG9ydGQABHRydWU.bkr_zPZ51MLOp-PZhwVEyN4ivpWE1d91TT5hgKCRzTA"

curl-iv \
       -X GET \
       -H "Accept: application/json;pesto" \
       -H "Content-Type: application/json;pesto" \
       -H "Authorization: Bearer ${PESTO_TOKEN}" \
       -H GET \
       http://plausible.pesto.io:8080/anything | jq .

I get the logs in OPA, rejecting my authirzation (because my token is absolutely not valid :

opa_1       | {
opa_1       |   "dry-run": false,
opa_1       |   "input": [
opa_1       |     [
opa_1       |       {
opa_1       |         "type": "string",
opa_1       |         "value": "parsed_path"
opa_1       |       },
opa_1       |       {
opa_1       |         "type": "array",
opa_1       |         "value": [
opa_1       |           {
opa_1       |             "type": "string",
opa_1       |             "value": "get"
opa_1       |           }
opa_1       |         ]
opa_1       |       }
opa_1       |     ],
opa_1       |     [
opa_1       |       {
opa_1       |         "type": "string",
opa_1       |         "value": "parsed_query"
opa_1       |       },
opa_1       |       {
opa_1       |         "type": "object",
opa_1       |         "value": []
opa_1       |       }
opa_1       |     ],
opa_1       |     [
opa_1       |       {
opa_1       |         "type": "string",
opa_1       |         "value": "parsed_body"
opa_1       |       },
opa_1       |       {
opa_1       |         "type": "null",
opa_1       |         "value": {}
opa_1       |       }
opa_1       |     ],
opa_1       |     [
opa_1       |       {
opa_1       |         "type": "string",
opa_1       |         "value": "truncated_body"
opa_1       |       },
opa_1       |       {
opa_1       |         "type": "boolean",
opa_1       |         "value": false
opa_1       |       }
opa_1       |     ],
opa_1       |     [
opa_1       |       {
opa_1       |         "type": "string",
opa_1       |         "value": "attributes"
opa_1       |       },
opa_1       |       {
opa_1       |         "type": "object",
opa_1       |         "value": [
opa_1       |           [
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "source"
opa_1       |             },
opa_1       |             {
opa_1       |               "type": "object",
opa_1       |               "value": [
opa_1       |                 [
opa_1       |                   {
opa_1       |                     "type": "string",
opa_1       |                     "value": "address"
opa_1       |                   },
opa_1       |                   {
opa_1       |                     "type": "object",
opa_1       |                     "value": [
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "socketAddress"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "object",
opa_1       |                           "value": [
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "address"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "192.168.24.236"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "portValue"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "number",
opa_1       |                                 "value": 11646
opa_1       |                               }
opa_1       |                             ]
opa_1       |                           ]
opa_1       |                         }
opa_1       |                       ]
opa_1       |                     ]
opa_1       |                   }
opa_1       |                 ]
opa_1       |               ]
opa_1       |             }
opa_1       |           ],
opa_1       |           [
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "destination"
opa_1       |             },
opa_1       |             {
opa_1       |               "type": "object",
opa_1       |               "value": [
opa_1       |                 [
opa_1       |                   {
opa_1       |                     "type": "string",
opa_1       |                     "value": "address"
opa_1       |                   },
opa_1       |                   {
opa_1       |                     "type": "object",
opa_1       |                     "value": [
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "socketAddress"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "object",
opa_1       |                           "value": [
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "address"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "172.21.0.2"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "portValue"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "number",
opa_1       |                                 "value": 80
opa_1       |                               }
opa_1       |                             ]
opa_1       |                           ]
opa_1       |                         }
opa_1       |                       ]
opa_1       |                     ]
opa_1       |                   }
opa_1       |                 ]
opa_1       |               ]
opa_1       |             }
opa_1       |           ],
opa_1       |           [
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "request"
opa_1       |             },
opa_1       |             {
opa_1       |               "type": "object",
opa_1       |               "value": [
opa_1       |                 [
opa_1       |                   {
opa_1       |                     "type": "string",
opa_1       |                     "value": "time"
opa_1       |                   },
opa_1       |                   {
opa_1       |                     "type": "string",
opa_1       |                     "value": "2023-11-04T09:26:12.526999Z"
opa_1       |                   }
opa_1       |                 ],
opa_1       |                 [
opa_1       |                   {
opa_1       |                     "type": "string",
opa_1       |                     "value": "http"
opa_1       |                   },
opa_1       |                   {
opa_1       |                     "type": "object",
opa_1       |                     "value": [
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "id"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "4936379084437054517"
opa_1       |                         }
opa_1       |                       ],
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "method"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "GET"
opa_1       |                         }
opa_1       |                       ],
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "headers"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "object",
opa_1       |                           "value": [
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "user-agent"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "curl/7.77.0"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "x-forwarded-proto"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "http"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": ":authority"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "plausible.pesto.io:8080"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": ":method"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "GET"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": ":path"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "/get"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "accept"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "application/json;pesto"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "authorization"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "Bearer SFMyNTY.g3QAAAAGbQAAAAtfY3NyZl90b2tlbm0AAAAYZ3RTay1EbGFtTE51cnkzcFlQd3gydjFkbQAAAA9jdXJyZW50X3VzZXJfaWRhAW0AAAAJbGFzdF9zZWVuYmVCM_ttAAAAJXBsYXVzaWJsZS5wZXN0by5pb19vZmZlcl9lbWFpbF9yZXBvcnRkAAR0cnVlbQAAABJzZXNzaW9uX3RpbWVvdXRfYXRiZVSrS20AAAAodGVzdC13ZWJzaXRlLnBva3VzLmlvX29mZmVyX2VtYWlsX3JlcG9ydGQABHRydWU.bkr_zPZ51MLOp-PZhwVEyN4ivpWE1d91TT5hgKCRzTA"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "content-type"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "application/json;pesto"
opa_1       |                               }
opa_1       |                             ],
opa_1       |                             [
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "x-request-id"
opa_1       |                               },
opa_1       |                               {
opa_1       |                                 "type": "string",
opa_1       |                                 "value": "e387506c-ec39-43de-8e60-f8415372eb57"
opa_1       |                               }
opa_1       |                             ]
opa_1       |                           ]
opa_1       |                         }
opa_1       |                       ],
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "path"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "/get"
opa_1       |                         }
opa_1       |                       ],
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "host"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "plausible.pesto.io:8080"
opa_1       |                         }
opa_1       |                       ],
opa_1       |                       [
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "protocol"
opa_1       |                         },
opa_1       |                         {
opa_1       |                           "type": "string",
opa_1       |                           "value": "HTTP/1.1"
opa_1       |                         }
opa_1       |                       ]
opa_1       |                     ]
opa_1       |                   }
opa_1       |                 ]
opa_1       |               ]
opa_1       |             }
opa_1       |           ],
opa_1       |           [
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "metadataContext"
opa_1       |             },
opa_1       |             {
opa_1       |               "type": "object",
opa_1       |               "value": []
opa_1       |             }
opa_1       |           ]
opa_1       |         ]
opa_1       |       }
opa_1       |     ],
opa_1       |     [
opa_1       |       {
opa_1       |         "type": "string",
opa_1       |         "value": "version"
opa_1       |       },
opa_1       |       {
opa_1       |         "type": "object",
opa_1       |         "value": [
opa_1       |           [
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "ext_authz"
opa_1       |             },
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "v3"
opa_1       |             }
opa_1       |           ],
opa_1       |           [
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "encoding"
opa_1       |             },
opa_1       |             {
opa_1       |               "type": "string",
opa_1       |               "value": "protojson"
opa_1       |             }
opa_1       |           ]
opa_1       |         ]
opa_1       |       }
opa_1       |     ]
opa_1       |   ],
opa_1       |   "level": "debug",
opa_1       |   "msg": "Executing policy query.",
opa_1       |   "query": "data.envoy.authz.allow",
opa_1       |   "time": "2023-11-04T09:26:12Z",
opa_1       |   "txn": 10
opa_1       | }
opa_1       | {
opa_1       |   "decision_id": "8dc24cd3-5e85-4df5-a8a7-252f9232171b",
opa_1       |   "error": {
opa_1       |     "message": "undefined decision"
opa_1       |   },
opa_1       |   "input": {
opa_1       |     "attributes": {
opa_1       |       "destination": {
opa_1       |         "address": {
opa_1       |           "socketAddress": {
opa_1       |             "address": "172.21.0.2",
opa_1       |             "portValue": 80
opa_1       |           }
opa_1       |         }
opa_1       |       },
opa_1       |       "metadataContext": {},
opa_1       |       "request": {
opa_1       |         "http": {
opa_1       |           "headers": {
opa_1       |             ":authority": "plausible.pesto.io:8080",
opa_1       |             ":method": "GET",
opa_1       |             ":path": "/get",
opa_1       |             "accept": "application/json;pesto",
opa_1       |             "authorization": "Bearer SFMyNTY.g3QAAAAGbQAAAAtfY3NyZl90b2tlbm0AAAAYZ3RTay1EbGFtTE51cnkzcFlQd3gydjFkbQAAAA9jdXJyZW50X3VzZXJfaWRhAW0AAAAJbGFzdF9zZWVuYmVCM_ttAAAAJXBsYXVzaWJsZS5wZXN0by5pb19vZmZlcl9lbWFpbF9yZXBvcnRkAAR0cnVlbQAAABJzZXNzaW9uX3RpbWVvdXRfYXRiZVSrS20AAAAodGVzdC13ZWJzaXRlLnBva3VzLmlvX29mZmVyX2VtYWlsX3JlcG9ydGQABHRydWU.bkr_zPZ51MLOp-PZhwVEyN4ivpWE1d91TT5hgKCRzTA",
opa_1       |             "content-type": "application/json;pesto",
opa_1       |             "user-agent": "curl/7.77.0",
opa_1       |             "x-forwarded-proto": "http",
opa_1       |             "x-request-id": "e387506c-ec39-43de-8e60-f8415372eb57"
opa_1       |           },
opa_1       |           "host": "plausible.pesto.io:8080",
opa_1       |           "id": "4936379084437054517",
opa_1       |           "method": "GET",
opa_1       |           "path": "/get",
opa_1       |           "protocol": "HTTP/1.1"
opa_1       |         },
opa_1       |         "time": "2023-11-04T09:26:12.526999Z"
opa_1       |       },
opa_1       |       "source": {
opa_1       |         "address": {
opa_1       |           "socketAddress": {
opa_1       |             "address": "192.168.24.236",
opa_1       |             "portValue": 11646
opa_1       |           }
opa_1       |         }
opa_1       |       }
opa_1       |     },
opa_1       |     "parsed_body": null,
opa_1       |     "parsed_path": [
opa_1       |       "get"
opa_1       |     ],
opa_1       |     "parsed_query": {},
opa_1       |     "truncated_body": false,
opa_1       |     "version": {
opa_1       |       "encoding": "protojson",
opa_1       |       "ext_authz": "v3"
opa_1       |     }
opa_1       |   },
opa_1       |   "labels": {
opa_1       |     "id": "1f6063e1-22c0-4032-a473-6304c5e15338",
opa_1       |     "version": "0.26.0-envoy"
opa_1       |   },
opa_1       |   "level": "info",
opa_1       |   "metrics": {
opa_1       |     "timer_rego_external_resolve_ns": 475,
opa_1       |     "timer_rego_query_eval_ns": 16622,
opa_1       |     "timer_server_handler_ns": 871422
opa_1       |   },
opa_1       |   "msg": "Decision Log",
opa_1       |   "path": "envoy/authz/allow",
opa_1       |   "requested_by": "",
opa_1       |   "time": "2023-11-04T09:26:12Z",
opa_1       |   "timestamp": "2023-11-04T09:26:12.528621985Z",
opa_1       |   "type": "openpolicyagent.org/decision_logs"
opa_1       | }

I just grabbed the yaml config in the envoy container, just to check, here :

jbl@pokus2:~/envoy-opa-compose$ docker cp envoy-opa-compose_envoy_1:/etc/envoy/envoy.yaml ./recup1_envoy.yaml
Successfully copied 3.58kB to /home/jbl/envoy-opa-compose/recup1_envoy.yaml
jbl@pokus2:~/envoy-opa-compose$ more ./recup1_envoy.yaml
admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      protocol: TCP
      address: 0.0.0.0
      port_value: 9901
static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 10000
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match:
                  prefix: "/"
                route:
                  host_rewrite_literal: www.envoyproxy.io
                  cluster: service_envoyproxy_io
          http_filters:
          - name: envoy.filters.http.router
  clusters:
  - name: service_envoyproxy_io
    connect_timeout: 30s
    type: LOGICAL_DNS
    # Comment out the following line to test on v6 networks
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: service_envoyproxy_io
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: www.envoyproxy.io
                port_value: 443
    transport_socket:
      name: envoy.transport_sockets.tls
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
        sni: www.envoyproxy.io

Ok, now, I 'm trying to get my hands on the opa configuration file. There we have a distroless container, but the same image with shell exists, for debug purposes :

image

Jean-Baptiste-Lasselle commented 11 months ago
# opa run -s -c config.yaml
# openpolicyagent/opa:<version>-debug
version: "3.7"
services:
  envoy:
    build: ./compose/envoy
    ports:
      - "8080:80"
    volumes:
      - ./envoy.yaml:/config/envoy.yaml
    environment:
      # - DEBUG_LEVEL=info
      - DEBUG_LEVEL=debug
      - SERVICE_NAME=app  # should match name of upstream service
      - SERVICE_PORT=80
    extra_hosts:
        - "envoy.pesto.io:192.168.24.202"
        - "keycloak.pesto.io:192.168.24.202"
        - "opa.pesto.io:192.168.24.202"
        - "app.pesto.io:192.168.24.202"
        - "testwebsite.pokus.io:192.168.24.236"
    networks:
      pesto_net:
        aliases:
          - envoy.pesto.io

  keycloak:
    image: "quay.io/keycloak/keycloak:19.0.1"
    command: 'start-dev'
    ports:
      - "8081:8080"
    # volumes:
    #  - ./envoy.yaml:/config/envoy.yaml
    environment:
      - KC_SERVER_URL=http://keycloak.pesto.io:8081/auth
      # - KC_SERVER_URL=http://localhost:8080/auth
      - KC_ADMIN_USER=admin
      - KC_ADMIN_PASSWORD=admin
      - KC_REALM=pesto
      - KEYCLOAK_ADMIN=pesto
      - KEYCLOAK_ADMIN_PASSWORD=pesto
      - KC_HTTP_RELATIVE_PATH=auth
      # - KC_DB=postgres
      # - KC_DB_URL=<DBURL>
      # - KC_DB_USERNAME=<DBUSERNAME>
      # - KC_DB_PASSWORD=<DBPASSWORD>
      - KC_HOSTNAME=keycloak.pesto.io
    extra_hosts:
        - "keycloak.pesto.io:192.168.24.202"
        - "testwebsite.pokus.io:192.168.24.236"
    networks:
      pesto_net:
        aliases:
          - keycloak.pesto.io

  opa:
    # image: openpolicyagent/opa:0.26.0-envoy
    image: openpolicyagent/opa:0.26.0-debug
    volumes:
      - ./policy.rego:/config/policy.rego
    command:
      - "run"
      - "--log-level=debug"
      - "--log-format=json-pretty"
      - "--server"
      - "--set=plugins.envoy_ext_authz_grpc.path=envoy/authz/allow"  # default value
      - "--set=decision_logs.console=true"
      - "/config/policy.rego"
    extra_hosts:
        - "envoy.pesto.io:192.168.24.202"
        - "keycloak.pesto.io:192.168.24.202"
        - "opa.pesto.io:192.168.24.202"
        - "app.pesto.io:192.168.24.202"
        - "testwebsite.pokus.io:192.168.24.236"
    networks:
      pesto_net:
        aliases:
          - opa.pesto.io
  app:
    image: kennethreitz/httpbin:latest
    extra_hosts:
        - "envoy.pesto.io:192.168.24.202"
        - "keycloak.pesto.io:192.168.24.202"
        - "opa.pesto.io:192.168.24.202"
        - "app.pesto.io:192.168.24.202"
        - "testwebsite.pokus.io:192.168.24.236"
    networks:
      pesto_net:
        aliases:
          - app.pesto.io

# --- --- ---  
#  + 
networks:
  pesto_net:
  pokus_net:
Jean-Baptiste-Lasselle commented 11 months ago

An Envoy Expert mentioning the external authentication filter etc/authz of Envoy.

He specifies it is possible to implement a cusom envoy filter using wasm ! awesome.