Permission problem when trying to authenticate api requests with envoy filter header set

[RegaliaXYZ]

I deployed a middleware on the same cluster as kubeflow (vanilla terraform deployment) and im trying to use the pipeline API. For that i created an authorization policy like below ` apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: allow-all namespace: kubeflow spec: rules:

• {} and an envoy filter like below apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: add-header namespace: kubeflow spec: configPatches:
• applyTo: VIRTUAL_HOST match: context: SIDECAR_OUTBOUND routeConfiguration: vhost: name: ml-pipeline-ui.kubeflow.svc.cluster.local:80 route: name: default patch: operation: MERGE value: request_headers_to_add:
  • append: false header: key: kubeflow-userid value: user@example.com workloadSelector: labels: app: app-kf-middleware ` note that the app-kf-middleware is the middleware trying to request kubeflow. In there i set headers to have kubeflow-userid set to user@example.com

And it works, but only for certain requests: http://ml-pipeline-ui.kubeflow/apis/v1beta1/runs/specific-run-id works but http://ml-pipeline-ui.kubeflow/apis/v1beta1/runs/ to get all runs fails with this error message { "code": 7, "details": [ { "@type": "type.googleapis.com/api.Error", "error_details": "Failed to authorize the request: Failed to authorize with API resource references: PermissionDenied: User 'user@example.com' is not authorized with reason: (request: &ResourceAttributes{Namespace:,Verb:get,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access", "error_message": "User 'user@example.com' is not authorized with reason: (request: &ResourceAttributes{Namespace:,Verb:get,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,})" } ], "error": "Failed to authorize the request: Failed to authorize with API resource references: PermissionDenied: User 'user@example.com' is not authorized with reason: (request: &ResourceAttributes{Namespace:,Verb:get,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access", "message": "Failed to authorize the request: Failed to authorize with API resource references: PermissionDenied: User 'user@example.com' is not authorized with reason: (request: &ResourceAttributes{Namespace:,Verb:get,Group:pipelines.kubeflow.org,Version:v1beta1,Resource:runs,Subresource:,Name:,}): Unauthorized access" }

From what i can tell the requests are correctly authenticated but the user@example.com created during the deployment lacks permissions, is there a way to give it every permission? Or is the problem different from what I thought?

[rrrkharse]

Hi RegaliaXYZ, have you authorized (logged in) the user and passed the auth token along with the request?

This should be similar to the steps for authenticating KFP client requests from outside the cluster. Docs

# This is the "Domain" in your cookies. Eg: "localhost:8080" or "<ingress_alb_address>.elb.amazonaws.com"
kubeflow_gateway_endpoint="<YOUR_KUBEFLOW_GATEWAY_ENDPOINT>"

authservice_session_cookie="<YOUR_COOKIE>"      # This cookie can be found in the session/response after logging in

namespace="<YOUR_NAMESPACE>"

client = kfp.Client(host=f"http://{kubeflow_gateway_endpoint}/pipeline", cookies=f"authservice_session={authservice_session_cookie}")
client.list_experiments(namespace=namespace)

[RegaliaXYZ]

Hello, since i'm not using python and therefore do not have access to the kfp package i cannot do that. On GCP what i used to do was create an authorization policy like this

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  annotations:
    role: edit
    user: userid@email.com # replace with the email of the user from your Active Directory case sensitive
  name: user-userid-email-com-clusterrole-edit
  namespace: profileName # replace with the namespace/profile name that you are adding contributors to
spec:
  action: ALLOW
  rules:
  - when:
    - key: x-goog-authenticated-user-email] 
      values:
      - accounts.google.com:userid@email.com   # REPLACED this with service account 

I tried a similar approach of replacing the key with request.headers[kubeflow-userid] however it doesnt seem to be working, or at least only partially as stated in my previous post. Im trying to replicate the behavior of allowing an external api middleware to communicate with kubeflow while on the same cluster with only envoyfilters/authorization policies

[rrrkharse]

I see, what I was trying to say was do you have an authorization token present in the request?

[RegaliaXYZ]

No i do not have any, i was using the header previously and I'm trying to replicate that way of doing things on aws