how to declare nested map in CRD yaml

[ffoysal]

Type of question

how to implement a specific feature, or about general context and help around the operator-sdk

Question

What did you do?

type MyJobSpec struct {
    // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
    // Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file
    // Add custom validation using kubebuilder tags: https://book-v1.book.kubebuilder.io/beyond_basics/generating_crd.html
    JobName       string `yaml:"jobName,omitempty"`
    Services     map[string]interface{} `yaml:"services,omitempty"`
}

expecting in custom resource yaml nested map can be defined

services:
  serv1:
    val: val3
    another:
      an_val: val4
.........

What did you expect to see? operator-sdk generate k8s should be successful

What did you see instead? Under which circumstances? it throws this exception

 4009 deepcopy.go:750] DeepCopy of "interface{}" is unsupported. Instead, use named interfaces with DeepCopy<named-interface> as one of the methods.

Environment

• operator-sdk version:

  operator-sdk version: v0.9.0, commit: 560208dc998de497bbf59fea1b63426aec430934

• Kubernetes version information:

  Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T14:25:20Z", GoVersion:"go1.12.7", Compiler:"gc", Platform:"darwin/amd64"}
  Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:36:19Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes cluster kind:

docker-desktop, minikube

Additional context Add any other context about the question here.

[camilamacedo86]

Hi @ffoysal,

I do not think that map[string]interface{} will work in this case. Instead of that, I think that you could use an object which represents what you would like to have. Something as:

type MyJobSpec struct {
         ...
    Services []Service `json:"services,omitempty"`
}

type Service struct {
    Name string `json:"name"`
    ID string `json:"id"`
}

Did you tried to solve your need doing something as above?

[ffoysal]

Thank you @camilamacedo86 yes I tried similar thing but did not solve my problem. What I would like to achieve is having a valid yaml document under services:. So it could be anything as long as it is valid yaml.

[camilamacedo86]

HI @ffoysal,

I am not sure if I understood what you mean with What I would like to achieve is having a valid YAML document under services:.

Note that when we are working with k8s and/or OCP by editing any YAML file what we actually do is edit the values of attributes defined for the objects which are defined in these API's. So, for example, a Deployment YAML file configuration is an Object from K8s which is defined in its API. See here. The YAML file is just a format for you are able to check/edit it.

In this way, in the CR's you are able to set the values of the specs(attributes) of your own object definition(CRD) of your own API. Following some examples.

• See here an example of one spec which is a K8S object.
• See here and here code of a spec which expected an object defined locally like the example suggested.
• Good example to check it working like what was suggested above.

I hope that it helps you with.

[ffoysal]

Thank you very much @camilamacedo86 for nice references. I am sorry for confusions, actually my intention to have valid yaml document under services: meaning to have similar kind of open yaml structure like helm values file for example https://github.com/helm/charts/blob/master/stable/mongodb/values.yaml. In other words under services: I will have any values that is supported by helm values file. As helm values file has an open structure similarly I dont have specific types what will go under services: for my task.

[joelanford]

@ffoysal

4009 deepcopy.go:750] DeepCopy of "interface{}" is unsupported. Instead, use named interfaces with DeepCopy<named-interface> as one of the methods.

This error message basically means that the deep copy generators don't know how to copy interface{}, which makes sense, because interface{} can be anything.

It's telling you to declare an interface (e.g. Values) that includes a DeepCopyValues() method in the interface definition. While that would probably work for deep copy generation, I think you'd run into problems with that approach for CRD generation.

Rather, I think you can probably declare a typed map[string]interface{} that has a DeepCopy() method (that you have to implement yourself).

type Values map[string]interface{}
func (v *Values) DeepCopy() *Values { ... }
// or func(v Values) DeepCopy() Values { ... }

[Finkes]

When I try the solution proposal I receive the following error: map values must be a named type, not *ast.InterfaceType

Any ideas whether this has changed? At least it sounds like this was a proper solution in 2019 :-)

[ccremer]

I'm also very interested to know. Previously, at least in K8s 1.18 we could do this:

SyncItems []unstructured.Unstructured `json:"syncItems,omitempty"`

with a spec like

  syncItems:
  - apiVersion: v1
    kind: ConfigMap
    metadata:
      name: test-data

But now that we want to upgrade our Operator to K8s 1.20, this doesn't work anymore (not sure if that would be already the case with 1.19). The property is empty:

  syncItems:
    - {}

This creates errors all over the place when Operator tries to read/deserialize the JSON from the server.

[ccremer]

Actually, we "sort of" figured it out. It's not a K8s version issue, rather one with the CRD version.

Using unstructured.Unstructured is fine, serialization and deserialization generally works with this type (cc @Finkes)

But, if the CRD is version apiextension.k8s.io/v1 (with v1beta1 it works out of the box), then the server removes unknown fields. This can be counteracted by setting // +kubebuilder:pruning:PreserveUnknownFields on that field. However, since clients also do validation, resources need to be applied with kubectl apply --validate=false...

(see https://book.kubebuilder.io/reference/markers/crd-processing.html)

[ccremer]

We made it work even with validatation:

import "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
type (
    someStruct struct {
        SyncItems []SyncItem `json:"syncItems,omitempty"`
    }
    // +kubebuilder:pruning:PreserveUnknownFields
    // +kubebuilder:validation:EmbeddedResource
    SyncItem unstructured.Unstructured
)

func (in *SyncItem) DeepCopyInto(out *SyncItem) {
    // controller-gen cannot handle the interface{} type of an aliased Unstructured, thus we write our own DeepCopyInto function.
    if out != nil {
        casted := unstructured.Unstructured(*in)
        deepCopy := casted.DeepCopy()
        out.Object = deepCopy.Object
    }
}