I can't authenticate and get a session

[yurkoff-mv]

I'm trying to get a session as per the following instructions: https://github.com/kserve/kserve/tree/master/docs/samples/istio-dex However, the responses are somewhat different.

1. CLUSTER_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.clusterIP}')

2. curl -v http://${CLUSTER_IP}

Response: >> <a href="/dex/auth?client_id=kubeflow-oidc-authservice&amp;redirect_uri=%2Fauthservice%2Foidc%2Fcallback&amp;response_type=code&amp;scope=openid+profile+email+groups&amp;state=MTY5NDc4ODI5N3xOd3dBTkVGRlRESTJSMGxhV1UwMU1qSkxSRWRDVEZWQ1RrUkNRa2xIUVVWRFZWUlFWVGRUUVVWS1NWVlNNek5EVlV4YVMwNVhXbEU9fGcVV1W5Jwh1NgnAzm1p02a5_n7gGeUS2sg2w3jN0gkY">Found</a>.

STATE=STATE_VALUE

3. curl -v "http://192.168.21.45:31368/dex/auth/local?client_id=kubeflow-oidc-authservice&redirect_uri=%2Fauthservice%2Foidc%2Fcallback&response_type=code&scope=profile+email+groups+openid&state=MTY5NDc4ODI5N3xOd3dBTkVGRlRESTJSMGxhV1UwMU1qSkxSRWRDVEZWQ1RrUkNRa2xIUVVWRFZWUlFWVGRUUVVWS1NWVlNNek5EVlV4YVMwNVhXbEU9fGcVV1W5Jwh1NgnAzm1p02a5_n7gGeUS2sg2w3jN0gkY"

Response: >><a href="/dex/auth/local/login?back=&amp;state=ijw5ccyzf4ujzcmaqjxsaurgl">Found</a>. But the response should be as follows: >> <a href="/dex/auth/local?req=REQ_VALUE">Found</a>

Please note that the link in the answer also does not correspond to the instructions: redirect_uri=%2Fauthservice%2Foidc%2Fcallback instead redirect_uri=%2Flogin%2Foidc

What did you expect to happen: I expected to receive a session according to the instructions.

Environment:

• Istio Version: 1.17.3
• Knative Version: v1.10.2
• KServe Version: v0.10
• Kubeflow version: 1.8
• Kubernetes version: (use kubectl version): v1.26
• OS (e.g. from /etc/os-release): Ubuntu 20.04

[juliusvonkohout]

Would you mind debugging it a bit? Because for me 1.8 is working so far.

[yurkoff-mv]

Please tell me how to debug this?

[juliusvonkohout]

Update to Kubeflow 1.8rc2 or the 1.8 branch first. Then check the logs of oidc-authservice and Dex.

[yurkoff-mv]

microk8s kubectl logs oidc-authservice-0 -n istio-system

time="2023-11-07T07:24:03Z" level=info msg="URI is whitelisted. Accepted without authorization." ip=10.1.251.226 request="/dex/auth?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc&response_type=code&scope=profile+email+groups+openid&state=MTY5OTM0MTA2MXxOd3dBTkZoYU5FcEJSMUJWUjBoQ1dWTlVTMGRGU1VoTVJreExVa1ZOVUZWTlRFUmFOVlV5VXpOT00xZFlSRlpQV1ZCWFQxWlFUa0U9fIKkWVRCdHPead9uMHoqZVJ56bPH9AvWADEw0iiDiqCG"

microk8s kubectl logs dex-6555448c78-tgvvv -n auth

time="2023-11-07T07:24:03Z" level=error msg="Failed to parse arguments: invalid semicolon separator in query"
2023/11/07 07:24:03 http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192

[yurkoff-mv]

You can try to get something similar to a session in the following way...

CLUSTER_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.clusterIP}')

curl -v http://${CLUSTER_IP}

Response:

>> <a href="/dex/auth?client_id=kubeflow-oidc-authservice&amp;redirect_uri=%2Fauthservice%2Foidc%2Fcallback&amp;response_type=code&amp;scope=openid+profile+email+groups&amp;state=MTY5NDc4ODI5N3xOd3dBTkVGRlRESTJSMGxhV1UwMU1qSkxSRWRDVEZWQ1RrUkNRa2xIUVVWRFZWUlFWVGRUUVVWS1NWVlNNek5EVlV4YVMwNVhXbEU9fGcVV1W5Jwh1NgnAzm1p02a5_n7gGeUS2sg2w3jN0gkY">Found</a>.

STATE=STATE_VALUE

But this is absolutely not what is described in the documentation. Can someone follow the path that is written in the documentation? Maybe it’s even worth returning to the previous version of Dex for normal authorization?

curl -v "${CLUSTER_IP}/dex/auth/local?client_id=kubeflow-oidc-authservice&redirect_uri=%2Fauthservice%2Foidc%2Fcallback&response_type=code&scope=profile+email+groups+openid&state=MTY5NDc4ODI5N3xOd3dBTkVGRlRESTJSMGxhV1UwMU1qSkxSRWRDVEZWQ1RrUkNRa2xIUVVWRFZWUlFWVGRUUVVWS1NWVlNNek5EVlV4YVMwNVhXbEU9fGcVV1W5Jwh1NgnAzm1p02a5_n7gGeUS2sg2w3jN0gkY"

Response: >><a href="/dex/auth/local/login?back=&amp;state=ijw5ccyzf4ujzcmaqjxsaurgl">Found</a>.

STATE=STATE_VALUE

curl -v "http://${CLUSTER_IP}/dex/auth/local/login?back=&state=${STATE}" Response:

'/authservice/oidc/callback?code=l7pndmotwybpqjwn3thkfzsrk&state=MTY5OTM0NzY2N3xOd3dBTkZaVlRqUlJXVFJLV1ZwTlIxUldXVTlhUzBOQ1NrcFRObEZYU1ZwUldGVkJWVUpOTWxjMFRWaE5SRmszUWtrMVJVZEZRbEU9fJMVRuAFy1Jq5aNkEUmk1ynt_xSzAEgeYLNrRq-ryqkY'

CODE=CODE_VALUE
STATE=STATE_VALUE

curl -v "http://${CLUSTER_IP}/login/oidc?code=${CODE}&state=${STATE}" Response:

>> set-cookie oidc_state_csrf=MTY5OTM0ODMzOHxOd3dBTkRKRFRVRkVURmxMTkVZMFRVSkZTelpPTkRjelRGcEpWVkZTTkZCR1FrcFJSbHBZVFVzek5UVlFSMGxWVlZSQ1QwSk1XRUU9fI-p1dkd2vKBa9ZIFC3x0han5zQiAahEe-BxCXJOdhCx

[yurkoff-mv]

I solved the problem. Here are the correct commands (semicolons and &amps are not needed)

1. CLUSTER_IP - ip-address and port of cluster in format x.x.x.x:x The port number can be obtained using the command: microk8s kubectl get service istio-ingressgateway -n istio-system Response:

   NAME                   TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                                                                      AGE
   istio-ingressgateway   NodePort   10.152.183.181   <none>        ...,80:PORT_VALUE/TCP,...   24h

   CLUSTER_IP=x.x.x.x:PORT_VALUE

2. curl http://${CLUSTER_IP} Response: <a href="/dex/auth?client_id=kubeflow-oidc-authservice&amp;redirect_uri=%2Flogin%2Foidc&amp;response_type=code&amp;scope=profile+email+groups+openid&amp;state=STATE_VALUE">Found</a>. STATE=STATE_VALUE

3. curl "http://${CLUSTER_IP}/dex/auth?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc&response_type=code&scope=profile+email+groups+openid&state=STATE" Response: Found. REQ=REQ_VALUE

4. curl "http://${CLUSTER_IP}/dex/auth/local?req=${REQ}" -H 'Content-Type: application/x-www-form-urlencoded' --data 'login=admin%40kubeflow.org&password=12341234'

5. curl "http://${CLUSTER_IP}/dex/approval?req=${REQ}" Response: <a href="/login/oidc?code=CODE_VALUE&amp;state=STATE_VALUE">See Other</a>. CODE=CODE_VALUE

6. curl -v "http://${CLUSTER_IP}/login/oidc?code=${CODE}&state=${STATE}" Response:

   < HTTP/1.1 302 Found
   < content-type: text/html; charset=utf-8
   < location: /
   < set-cookie: authservice_session=SESSION_VALUE; Path=/; Expires=Thu, 16 Nov 2023 18:51:47 GMT; Max-Age=86400
   < date: Wed, 15 Nov 2023 18:51:47 GMT
   < content-length: 24
   < x-envoy-upstream-service-time: 103
   < server: istio-envoy
   < 
   <a href="/">Found</a>.

[yurkoff-mv]

No, the problem still remains. The solution above was for version 1.5. Please try the session and share your solution.

[juliusvonkohout]

My current solution is to use the proper m2m authentication and not abusing human sessions. We will also switch to oauth2 soon. Feel free to help out in https://github.com/kubeflow/manifests/pull/2544

[yurkoff-mv]

Thank you very much! The authentication problem has finally been resolved. Here is a link to the get_istio_auth_session function to authenticate in the current version. I think that the documentation in the repository will be updated soon.

[jaffe-fly]

in kubeflow 1,9, in the get_istio_auth_session function, _auth_session["authservice_session"] is None.my _auth_session is

{'endpoint_url': 'http://xxxxxxx:18080', 
'redirect_url': 'http://xxxxxxx:18080/dex/auth/local/login?back=&state=sl6amu5zckyntv35z65solp4t',
'dex_login_url': 'http://xxxxxxx:18080/dex/auth/local/login?back=&state=sl6amu5zckyntv35z65solp4t', 
'is_secured': True, 
'session_cookie': 'oauth2_proxy_kubeflow_csrf=-GOrJQeg-v5_Dc_3HD12Td80HAuE9ld2eT3Y7dT2lyw1b2wUKEZaYJVjhqaBF7UW_elSQ8i08WsjXTGO2kys6VTPJWcHQ2uI_qMFqFp7Wn2_SIpjihvsfa-JX8VC-8Qag25f0jkbi-LMSu0IlRjVYoQZ2VDkriRa3bCq8j3zVOboKjcxVGujyrggNU1X4r-PyYVOujmdv7oxhw1wcS2HbYiQ84GZxPbb0S9FeFI8u2gkSvT5U_HQrpfl2MkoBg==|1723645690|pZzKszJaoId5Vb5vQN0-a99zY29WdQ2TfHtHdsIga-U=', 
'authservice_session': None}, 

use this cookies,still cant request get inferenceservices in kubeflow cluster,so how to fix this?

[yurkoff-mv]

I haven't upgraded to version 1.9 yet, in part because of possible problems like this. Please tell me, does the received token in the oauth2_proxy_kubeflow_csrf field not work (-GOrJQeg-v5_Dc_3HD12Td80HAuE9ld2eT3Y7dT2lyw1b2wUKEZaYJVjhqaBF7UW_elSQ8i08WsjXTGO2kys6VTPJWcHQ2uI_qMFqFp7Wn2_SIpjihvsfa-JX8VC-8Qag25f0jkbi-LMSu0IlRjVYoQZ2VDk riRa3bCq8j3zVOboKjcxVGujyrggNU1X4r-PyYVOujmdv7oxhw1wcS2HbYiQ84GZxPbb0S9FeFI8u2gkSvT5U_HQrpfl2MkoBg==|1723645690|pZzKszJaoId5Vb5vQN0-a99zY29WdQ2TfHtHdsIga-U= )? I mean if you get it again and try to use it.

[juliusvonkohout]

In general do not abuse sessions with usernames and passwords for programmatic access. Do not. We won't support it and won't help.

Oauth2-proxy and serviceaccountokens is the way to go. You can read the documentation here in our repository for aouth2-proxy.

The instructions are in

1. https://github.com/kubeflow/manifests/blob/master/.github/workflows/pipeline_run_from_notebook.yaml for jupyterlab in your namespace (inside cluster) to KFP Apiserver. 2.. The instructions for internal AND external access with the same power as the webinterface via oauth2-proxy and tokens are here for KFP onsole.cloud.google.com/ here for workbenches/notebooks https://github.com/kubeflow/manifests/blob/master/.github/workflows/notebook_controller_m2m_test.yaml and here for Kserve https://github.com/kubeflow/manifests/blob/master/.github/workflows/kserve_m2m_test.yaml It might be enough to just explain the general approach for KFP.

Afterwards all insecure/dangerous username+password based programatic access documentation on the website should be removed. These are anyway only hacky workarounds that try to emulate a webbrowser instead of providing a proper solution.

Kserve inference services external access is a separate topic. Please check the other issues in this repository for Kserve.

[juliusvonkohout]

Please reopen a new issue if needed.

[juliusvonkohout]

See https://github.com/kubeflow/manifests/issues/2811