create_from_yaml custom resource

[brantlk]

I've got a custom resource description and I need to create custom resource for it. I've been able to create it using client.CustomObjectsApi(api_client).create_namespaced_custom_object(...) and using kubectl apply -f. I tried using the utils.create_from_yaml but it fails with a traceback:

    utils.create_from_yaml(self._api_client, f.name)
  File "/usr/lib/python3.6/site-packages/kubernetes/utils/create_from_yaml.py", line 53, in create_from_yaml
    k8s_api = getattr(client, fcn_to_call)(k8s_client)
AttributeError: module 'kubernetes.client' has no attribute 'Brantlk.comV1Api'

yaml is like:

apiVersion: brantlk.com/v1
kind: MyCustomResource
metadata:
    name: brant-instance
    namespace: default
spec:
    repositories: []

Pretty obvious what the problem is from the code.

[micw523]

/assign We did not take CRD into consideration when the create_from_yaml function is rolled out. I'll need to investigate a little about how kubectl handles this and then will get back to you.

As a side note that the create from yaml function does not have apply -f capability but only create -f.

[govindKAG]

is there some kind of temporary workaround for this?

[rSrkn]

We are exactly having the same issue where we need to"Apply" some CRDs via Python K8s client. As @govindKAG mentioned, is there any temporary workaround that you could suggest?

[govindKAG]

@rSrkn Even the official API doesn't have a clean way to do the apply action. This stack overflow post might help you understand what the apply action is actually doing. I didn't need to apply anything myself so I ended up going with ruamel.yaml to parse the yaml file and just used the api methods.

[micw523]

Hi @rSrkn, there are talks of moving kubectl apply onto the server side so that we won't we supporting it in this client. If you're talking about just creating it is planned...

[rSrkn]

@micw523 Thanks for the info. We use ISTIO for traffic management and would like to manage/create/edit CRD resources (VirtualService, Gateway ...) from Python client. create_from_yaml is the only way I can think of for now for creating the CRD resource. (Which requires us to generate the yaml file in python.) Although here is not the best place to ask this but still related to this issue; Does anybody know better way or suggestion to manage CRDs resources (VirtualService, Gateway ...) ?

[govindKAG]

@rSrkn you can write a sort of template yaml and then use the yaml python package (I recommend ruamel.yaml though) to edit the yaml file to tailor it to your specific case and then use the appropriate API method, passing this edited yaml object as the "body" parameter for the method as a quick solution.

[micw523]

Is it possible that you use kubectl to create the yaml file / object? To create the custom resource definition, you can try the extensions API but there’s a big catch #376. create_from_yaml should do the job for you but you still need to be aware of the catch... To create a CRD object, you can try what the author of the PR suggests.

[rSrkn]

@micw523 Thank you , I will check the PR. Let me give little info about what we try to achieve. We run a web api (Python/Flask) inside a pod. This api reads/updates ISTIO CRDs on other namespaces (To show list of current domains, changes the domain names/ports ...) We would like to use only python client. Using command line, kubctl directly is not an option. For example to able to get VirtualService CRD we use below:

       # got the idea from apps_v1beta1_api.py
        response: HTTPResponse = self.api_client.call_api(
            '/apis/networking.istio.io/v1alpha3/namespaces/{namespace}/virtualservices', 'GET',
            path_params,
            query_params,
            header_params,
            body=body_params,
            post_params=form_params,
            files=local_var_files,
            response_type=None,
            auth_settings=auth_settings,
            async_req=params.get('async_req'),
            _return_http_data_only=params.get('_return_http_data_only'),
            _preload_content=params.get('_preload_content', True),
            _request_timeout=params.get('_request_timeout'),
            collection_formats=collection_formats
        )

which returns a urllib3.response.HTTPResponse. We parse it and convert into a VirtualService object (which we defined). So reading is easy. But for writing (Create/Patch), I can not say it is that straight forward. I am still digging in the source code. What I try to achieve here is to get / list / patch a CRDs like VirtualService object via Python K8s.Client.

[micw523]

Sorry, I meant the author of the issue sugggests*

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[cogfor]

/remove-lifecycle stale

Is there any progress on this? Any workaround that works?

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[eestewart]

/remove-lifecycle stale

[thiagosantosleite]

it would be very useful to me too.

[Frankkkkk]

I ~had~ have the same problem two days ago (thiagosantosleite's message reminded me of it). Would be useful for me too.

[thiagosantosleite]

my workaround was:

import yaml
from kubernetes import client

ISTIO_API='networking.istio.io'
ISTIO_API_VERSION='v1alpha3'

aConfiguration = client.Configuration()
aConfiguration.host = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
aConfiguration.api_key = {"authorization": "Bearer xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"}
aConfiguration.verify_ssl = False
clientapi = client.ApiClient(aConfiguration)
customObjectApi = client.CustomObjectsApi(clientapi)

yaml_data= """
{"apiVersion": "networking.istio.io/v1alpha3", "kind": "Gateway", "metadata": {"name": "gateway-xxxxxxxx", "namespace": "default"}, "spec": {"selector": {"istio": "ingressgateway"}, "servers": [{"port": {"number": 49999, "name": "tcp-49999", "protocol": "tcp"}, "hosts": ["*"]}]}}
"""

customObjectApi.create_namespaced_custom_object(ISTIO_API, ISTIO_API_VERSION, 'default', 'gateways', yaml.load(yaml_data,Loader=yaml.FullLoader))

But still, would be great if create_from_yaml works with custom objects :)

[Frankkkkk]

Thanks; way better than mine's

101         # XXX quick and ugly fallback                                           
102         # XXX this is really dangerous for now !                                
103         # XXX TODO change that before prod (or at least sanitize the yaml file)
104         cmd = ['kubectl', 'apply', '-n', namespace.name, '-f', yaml_file]                                                                                                                                                                                                                                                                                                                                                           
105         environ = os.environ.copy()                                             
106         environ['KUBECONFIG'] = cluster._get_kubeconfig_file()                  
107         subprocess.call(cmd, env=environ)  # 🤮🤮🤮                             

[micw523]

/lifecycle frozen In progress to change the backend to the dynamic client so that we can fit custom resources.

[arghya18]

Is there any better workaround or progress on the issue?

[molejnik-mergebit]

Would appreciate any update or ETA on this as well

[chenrulongmaster]

Here is my solution:

def get_custom_api() -> CustomObjectsApi:
    """
    获取custom api
    :return:
    """
    global custom_api
    if custom_api is None:
        custom_api = client.CustomObjectsApi(api_client)
    return custom_api

def patch_custom_object_from_yaml(yaml_object: dict, group: str, version: str, namespace: str, name: str, plural: str):
    """
    从yaml文件创建
    :param yaml_object:
    :param group:
    :param version:
    :param namespace:
    :param name:
    :param plural:
    :return:
    """
    return get_custom_api().patch_namespaced_custom_object(group,
                                                           version,
                                                           namespace,
                                                           plural,
                                                           name,
                                                           yaml_object)

def create_custom_object_from_yaml(yaml_object: dict, group: str, version: str, namespace: str, plural: str):
    """
    从yaml文件patch
    :param yaml_object:
    :param group:
    :param version:
    :param namespace:
    :param plural:
    :return:
    """
    return get_custom_api().create_namespaced_custom_object(group,
                                                            version,
                                                            namespace,
                                                            plural,
                                                            yaml_object)

def apply_custom_object_from_yaml(yaml_object: dict,
                                  group: str = None,
                                  version: str = None,
                                  namespace: str = None,
                                  name: str = None,
                                  plural: str = None):
    """
    apply yaml对象，创建或者更新
    :param yaml_object:
    :param group:
    :param version:
    :param namespace:
    :param name:
    :param plural:
    :return:
    """
    if not name:
        name = yaml_object['metadata']['name']
    if not group or not version:
        api_version = yaml_object['apiVersion']
        group = api_version[0: api_version.find('/')]
        version = api_version[api_version.find('/') + 1:]
    if not namespace:
        namespace = yaml_object['metadata']['namespace']
    if not plural:
        plural = yaml_object['kind'].lower() + 's'
    try:
        exists_obj = get_custom_api().get_namespaced_custom_object(group, version, namespace, plural, name)
        # 存在，进行patch
        patch_custom_object_from_yaml(yaml_object, group, version, namespace, name, plural)
    except ApiException as e:
        if e.status == 404:
            create_custom_object_from_yaml(yaml_object, group, version, namespace, plural)
        else:
            raise e

from the outside, use below codes to apply your yaml file

import yaml
yaml_text = 'xxxxx'
obj = yaml.load(yaml_text, yaml.CLoader)

apply_custom_object_from_yaml(obj)

[dingyiyi0226]

Is there any update to use dynamic client in create_from_yaml?

[doronl]

I don't see updates on this issue... according to k8s documentation the python client should support crds...

[BackMountainDevil]

I don't see updates on this issue... according to k8s documentation the python client should support crds...

https://github.com/kubernetes-client/python/blob/master/examples/dynamic-client/cluster_scoped_custom_resource.py