RBAC access denied on using Kubeflow SDK to create Pipeline

[delonx]

Notebook Which notebook you have problem with? 01_03_Notebook_Development_Pipeline

Describe the bug A clear and concise description of what the bug is. error on create_experiment() step of notebook

ERROR:root:Failed to read a token from file '/var/run/secrets/kubeflow/pipelines/token' ([Errno 2] No such file or directory: '/var/run/secrets/kubeflow/pipelines/token').
WARNING:root:Failed to set up default credentials. Proceeding without credentials...
ERROR:root:Failed to get healthz info attempt 1 of 5.
Traceback (most recent call last):
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp/_client.py", line 363, in get_kfp_healthz
    response = self._healthz_api.get_healthz()
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/api/healthz_service_api.py", line 63, in get_healthz
    return self.get_healthz_with_http_info(**kwargs)  # noqa: E501
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/api/healthz_service_api.py", line 148, in get_healthz_with_http_info
    collection_formats=collection_formats)
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/api_client.py", line 369, in call_api
    _preload_content, _request_timeout, _host)
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/api_client.py", line 188, in __call_api
    raise e
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/api_client.py", line 185, in __call_api
    _request_timeout=_request_timeout)
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/api_client.py", line 393, in request
    headers=headers)
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/rest.py", line 234, in GET
    query_params=query_params)
  File "/home/jovyan/.local/lib/python3.6/site-packages/kfp_server_api/rest.py", line 224, in request
    raise ApiException(http_resp=r)
kfp_server_api.exceptions.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'content-length': '19', 'content-type': 'text/plain', 'date': 'Mon, 26 Jul 2021 16:11:12 GMT', 'server': 'envoy', 'x-envoy-upstream-service-time': '1'})
HTTP response body: RBAC: access denied

Expected behavior A clear and concise description of what you expected to happen. create experiment successfully

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• EKS version: 1.17
• Kubeflow version: latest
• eks-kubeflow-workshop version: latest (26 july 2021)

[delonx]

I fixed it by updating the cell to use:

# Get or create an experiment and submit a pipeline run
EXPERIMENT_NAME='add_operation'
authservice_session='authservice_session=<cookie>'
client = kfp.Client(host='http://<alb_domain>/pipeline', cookies=authservice_session)
experiment = client.create_experiment(EXPERIMENT_NAME, namespace="admin")

this is inspired by first part of https://www.kubeflow.org/docs/distributions/aws/pipeline/

see results: