How to run pipeline from the cluster's outside

[yesid-lopez]

Hello, I did the setup of charmed kubeflow and it has worked pretty good so far, now I want to be able to build my own pipeline from the outside of the cluster but when I try to run the following code it does not work:

kfp_client = Client(host="http://10.64.140.43.nip.io")
kfp_client.list_experiments()

It lists empty experiments:

{'experiments': None, 'next_page_token': None, 'total_size': None}

When I run it from inside the notebook of kubeflow it works without the host argument.

kfp_client = Client()
kfp_client.list_experiments()

lists all the experiments

{'experiments': [{'created_at': datetime.datetime(2023, 5, 2, 8, 50, 23, tzinfo=tzlocal()),
                  'description': None,
                  'id': '2631a743-86f2-416b-8e86-22c24990df6b',
                  'name': 'demo-experiment',
                  'resource_references': [{'key': {'id': '35up',
                                                   'type': 'NAMESPACE'},
                                           'name': None,
                                           'relationship': 'OWNER'}],

[i-chvets]

Currently, it is not supported feature, because in order to connect to KFP from outside of the cluster authentication is required. Refer to the following link that migh be helpful in running pipeline from outside of cluster: https://www.kubeflow.org/docs/components/pipelines/v1/sdk/connect-api/#example-for-dex

[DnPlas]

Hi @yesid-lopez, thanks for raising this question. While it is true that from within the cluster you could just instantiate a kfp Client and use it for interacting with pipeline objects, this is not really applicable from outside the cluster as your Charmed Kubeflow deployment has authentication processes and may have more than one user, so you need to follow a different process. I suggest you follow this guide, give it a try and let us know if it works for you.