feat: predefined configurableInterpreter

[chaunceyjiang]

Signed-off-by: chaunceyjiang chaunceyjiang@gmail.com

What type of PR is this? /kind feature

What this PR does / why we need it:

Predefine ConfigurableInterpreter for some well-known third-party projects.

Which issue(s) this PR fixes: Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Predefine ConfigurableInterpreter for some well-known third-party projects.

[chaunceyjiang]

/cc @ikaven1024 @RainbowMango @XiShanYongYe-Chang

What do you think?

[ikaven1024]

I do't understand what scene need this feature. Could you show a user case?

[chaunceyjiang]

I do't understand what scene need this feature. Could you show a user case?

Currently, the built-in interpreter only maintains K8S resources. For some well-known third-party projects, Karmada may need to do some maintenance to reduce the cost of using the ConfigurableInterpreter.

For example, AggregateStatus may be a commonly used InterpreterOperation

@ikaven1024

[ikaven1024]

I do't understand what scene need this feature. Could you show a user case?

Currently, the built-in interpreter only maintains K8S resources. For some well-known third-party projects, Karmada may need to do some maintenance to reduce the cost of using the ConfigurableInterpreter.

For example, AggregateStatus may be a commonly used InterpreterOperation

This interpreters are supplied by karmada, then why not implement them with go, just like built-in, rather than inefficient lua

[chaunceyjiang]

why not implement them with go, just like built-in

Because if you use go to implement it, you need to introduce third-party project APIs, which will lead to more and more dependencies being introduced into Karmada.

[ikaven1024]

why not implement them with go, just like built-in

Because if you use go to implement it, you need to introduce third-party project APIs, which will lead to more and more dependencies being introduced into Karmada.

We can use unstructed, no need import real APIs

[chaunceyjiang]

We can use unstructed

This requires the maintainer of the third-party project to be familiar with the code of Karmada. On the other hand, this mechanism is convenient for the maintainer of the third-party project, not the maintainer of Kamrada.

[ikaven1024]

We can use unstructed

This requires the maintainer of the third-party project to be familiar with the code of Karmada. On the other hand, this mechanism is convenient for the maintainer of the third-party project, not the maintainer of Kamrada.

That's say, these script files are committed by third-party project.

[chaunceyjiang]

That's say, these script files are committed by third-party project.

Yes, Karmada maintainer or third-party project maintainer.

[Poor12]

Hi @chaunceyjiang, would you like to keep this pr going? We have plans to integrate with some well-known projects. I think this pr is needed.

[chaunceyjiang]

Hi @chaunceyjiang, would you like to keep this pr going?

Sure

[chaunceyjiang]

Directory Structure

``` resource_customizations ├── embed.go ├── webapp.my.domain # Group │ └── v1 # Version │ └── Guestbook # Kind │ ├── customizations.yaml # Fixed name │ ├── customizations_test.yaml │ └── testdata └── webapp2.my.domain └── v1 └── Guestbook ├── customizations.yaml ├── customizations_test.yaml └── testdata ```

customizations.yaml

``` # cat customizations.yaml apiVersion: config.karmada.io/v1alpha1 kind: ResourceInterpreterCustomization metadata: name: customization spec: target: apiVersion: apps/v1 kind: Deployment customizations: retention: luaScript: > function Retain(desiredObj, runtimeObj) desiredObj.metadata.annotations.cluster = runtimeObj.metadata.annotations.cluster return desiredObj end replicaResource: luaScript: > function GetReplicas(obj) replica = obj.spec.replicas requirement = { resourceRequest = obj.spec.template.spec.containers[1].resources.limits, nodeClaim = { nodeSelector = obj.spec.template.spec.nodeSelector, tolerations = obj.spec.template.spec.tolerations } } return replica, requirement end replicaRevision: luaScript: > function ReviseReplica(obj, desiredReplica) obj.spec.replicas = desiredReplica return obj end statusReflection: luaScript: > function ReflectStatus(observedObj) return observedObj.status end statusAggregation: luaScript: > function AggregateStatus(desiredObj, items) desiredObj.status.readyReplicas = 0 for i = 1, #items do desiredObj.status.readyReplicas = desiredObj.status.readyReplicas + items[i].status.readyReplicas end return desiredObj end healthInterpretation: luaScript: > function InterpretHealth(observedObj) return observedObj.status.readyReplicas == observedObj.spec.replicas end dependencyInterpretation: luaScript: > function GetDependencies(desiredObj) dependencies = {} dependencies[1] = { apiVersion = "v1", kind = "ServiceAccount", name = desiredObj.metadata.name, namespace = desiredObj.metadata.namespace, } return dependencies end --- apiVersion: config.karmada.io/v1alpha1 kind: ResourceInterpreterCustomization metadata: name: customization-2 spec: target: apiVersion: apps/v1 kind: Deployment customizations: dependencyInterpretation: luaScript: > function GetDependencies(desiredObj) dependencies = {} dependencies[1] = { apiVersion = "v1", kind = "ConfigMap", name = desiredObj.metadata.name, namespace = desiredObj.metadata.namespace, } return dependencies end --- ```

Q: Why is such a directory structure set up, and what are the benefits of doing so? A: First, it is easier to classify ResourceInterpreterCustomization for easy reference. Second, we can verify the content in customizations.yaml, just like karmada-webhook.

[chaunceyjiang]

/cc @Poor12 Please take a look.

[codecov-commenter]

Codecov Report

Merging #2768 (37b1e3f) into master (e4f5b62) will increase coverage by 0.17%. The diff coverage is 100.00%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##           master    #2768      +/-   ##
==========================================
+ Coverage   49.15%   49.32%   +0.17%     
==========================================
  Files         206      205       -1     
  Lines       18377    18274     -103     
==========================================
- Hits         9033     9014      -19     
+ Misses       8857     8778      -79     
+ Partials      487      482       -5     

Flag	Coverage Δ
unittests	49.32% <100.00%> (+0.17%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/karmadactl/interpret/check.go	91.17% <ø> (ø)
pkg/resourceinterpreter/configurable/luavm/kube.go	67.39% <ø> (ø)
pkg/resourceinterpreter/configurable/luavm/lua.go	57.14% <ø> (ø)
...ourceinterpreter/configurable/luavm/lua_convert.go	73.91% <ø> (ø)
...webhook/resourceinterpretercustomization/helper.go	82.05% <ø> (ø)
...r/configurableinterpreter/configmanager/manager.go	67.21% <100.00%> (ø)

... and 3 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[chaunceyjiang]

Test

I have not created the Interpretercustomizations resource, but you can see that the cloneset has the aggregatedStatus .

[chaunceyjiang]

Yaml template

``` apiVersion: config.karmada.io/v1alpha1 kind: ResourceInterpreterCustomization metadata: name: ric-cloneset spec: target: apiVersion: apps.kruise.io/v1alpha1 kind: CloneSet customizations: replicaResource: luaScript: > function GetReplicas(obj) replica = obj.spec.replicas requirement = {} return replica, requirement end replicaRevision: luaScript: > function ReviseReplica(obj, desiredReplica) obj.spec.replicas = desiredReplica return obj end retention: luaScript: > function Retain(desiredObj, observedObj) desiredObj.spec.paused = observedObj.spec.paused return desiredObj end statusAggregation: luaScript: > function AggregateStatus(desiredObj, statusItems) if statusItems == nil then return desiredObj end if desiredObj.status == nil then desiredObj.status = {} end replicas = 0 availableReplicas = 0 readyReplicas = 0 updatedReadyReplicas = 0 updatedReplicas = 0 for i = 1, #statusItems do if statusItems[i].status ~= nil and statusItems[i].status.replicas ~= nil then replicas = replicas + statusItems[i].status.replicas end if statusItems[i].status ~= nil and statusItems[i].status.availableReplicas ~= nil then availableReplicas = availableReplicas + statusItems[i].status.availableReplicas end if statusItems[i].status ~= nil and statusItems[i].status.readyReplicas ~= nil then readyReplicas = readyReplicas + statusItems[i].status.readyReplicas end if statusItems[i].status ~= nil and statusItems[i].status.updatedReadyReplicas ~= nil then updatedReadyReplicas = updatedReadyReplicas + statusItems[i].status.updatedReadyReplicas end if statusItems[i].status ~= nil and statusItems[i].status.updatedReplicas ~= nil then updatedReplicas = updatedReplicas + statusItems[i].status.updatedReplicas end end desiredObj.status.replicas = replicas desiredObj.status.availableReplicas = availableReplicas desiredObj.status.readyReplicas = readyReplicas desiredObj.status.updatedReadyReplicas = updatedReadyReplicas desiredObj.status.updatedReplicas = updatedReplicas return desiredObj end statusReflection: luaScript: > function ReflectStatus (observedObj) return observedObj.status end healthInterpretation: luaScript: > function InterpretHealth(observedObj) return observedObj.status.readyReplicas == observedObj.spec.replicas end dependencyInterpretation: luaScript: > function GetDependencies(desiredObj) dependentSas = {} refs = {} if desiredObj.spec.template.spec.serviceAccountName ~= '' and desiredObj.spec.template.spec.serviceAccountName ~= 'default' then dependentSas[desiredObj.spec.template.spec.serviceAccountName] = true end local idx = 1 for key, value in pairs(dependentSas) do dependObj = {} dependObj.apiVersion = 'v1' dependObj.kind = 'ServiceAccount' dependObj.name = key dependObj.namespace = desiredObj.metadata.namespace refs[idx] = dependObj idx = idx + 1 end return refs end --- apiVersion: config.karmada.io/v1alpha1 kind: ResourceInterpreterCustomization metadata: name: ric-cloneset spec: target: apiVersion: apps.kruise.io/v1alpha1 kind: CloneSet customizations: replicaResource: luaScript: > function GetReplicas(obj) replica = obj.spec.replicas requirement = {} return replica, requirement end replicaRevision: luaScript: > function ReviseReplica(obj, desiredReplica) obj.spec.replicas = desiredReplica return obj end retention: luaScript: > function Retain(desiredObj, observedObj) desiredObj.spec.paused = observedObj.spec.paused return desiredObj end statusAggregation: luaScript: > function AggregateStatus(desiredObj, statusItems) if statusItems == nil then return desiredObj end if desiredObj.status == nil then desiredObj.status = {} end replicas = 0 availableReplicas = 0 readyReplicas = 0 updatedReadyReplicas = 0 updatedReplicas = 0 for i = 1, #statusItems do if statusItems[i].status ~= nil and statusItems[i].status.replicas ~= nil then replicas = replicas + statusItems[i].status.replicas end if statusItems[i].status ~= nil and statusItems[i].status.availableReplicas ~= nil then availableReplicas = availableReplicas + statusItems[i].status.availableReplicas end if statusItems[i].status ~= nil and statusItems[i].status.readyReplicas ~= nil then readyReplicas = readyReplicas + statusItems[i].status.readyReplicas end if statusItems[i].status ~= nil and statusItems[i].status.updatedReadyReplicas ~= nil then updatedReadyReplicas = updatedReadyReplicas + statusItems[i].status.updatedReadyReplicas end if statusItems[i].status ~= nil and statusItems[i].status.updatedReplicas ~= nil then updatedReplicas = updatedReplicas + statusItems[i].status.updatedReplicas end end desiredObj.status.replicas = replicas desiredObj.status.availableReplicas = availableReplicas desiredObj.status.readyReplicas = readyReplicas desiredObj.status.updatedReadyReplicas = updatedReadyReplicas desiredObj.status.updatedReplicas = updatedReplicas return desiredObj end statusReflection: luaScript: > function ReflectStatus (observedObj) return observedObj.status end healthInterpretation: luaScript: > function InterpretHealth(observedObj) return observedObj.status.readyReplicas == observedObj.spec.replicas end dependencyInterpretation: luaScript: > function GetDependencies(desiredObj) dependentSas = {} refs = {} if desiredObj.spec.template.spec.serviceAccountName ~= '' and desiredObj.spec.template.spec.serviceAccountName ~= 'default' then dependentSas[desiredObj.spec.template.spec.serviceAccountName] = true end local idx = 1 for key, value in pairs(dependentSas) do dependObj = {} dependObj.apiVersion = 'v1' dependObj.kind = 'ServiceAccount' dependObj.name = key dependObj.namespace = desiredObj.metadata.namespace refs[idx] = dependObj idx = idx + 1 end return refs end ```

I have 1000 YAML files, totaling 7.8 MB in size. When I embedded these files into the karmada-controller-manager executable file, it only increased its file size by about 1 MB.

[RainbowMango]

I have 1000 YAML files, totaling 7.8 MB in size. When I embedded these files into the karmada-controller-manager executable file, it only increased its file size by about 1 MB.

Does 1000 YAML files mean 1000 resources?

[XiShanYongYe-Chang]

Can we organize the code like this? First PR focus on the code structure and let's back to this PR to abstract the common code and new features.

It makes sense.

[chaunceyjiang]

Does 1000 YAML files mean 1000 resources?

No, there will be some YAML files for testing one type of resource, and each type of resource has about 10 test files. So there are about 100 types of resources.

[RainbowMango]

No, there will be some YAML files for testing one type of resource, and each type of resource has about 10 test files. So there are about 100 types of resources.

100 resources VS 1MB, is totally acceptable I guess.

[chaunceyjiang]

100 resources VS 1MB, is totally acceptable I guess.

yes, i think so.

[chaunceyjiang]

Can we organize the code like this?

Can the luavm directory be separated from configureinterpreter?

[RainbowMango]

Can the luavm directory be separated from configureinterpreter?

I think keeping the luavm in pkg/resourceinterpreter/customized/declarative would be fine, at least it won't block this PR. Let's revisit during or after this PR.

[Poor12]

Good job! Could you please post the latest test result(whether the built-in configurableInterpret takes effect and whether users can override the default third-party crds)?

[chaunceyjiang]

build-in third-party configurableInterpreter:

---

users can override the third-party configurableInterpreter:

    statusAggregation:
      luaScript: >
        function AggregateStatus(desiredObj, statusItems)
          if statusItems == nil then
            return desiredObj
          end
          if desiredObj.status == nil then
            desiredObj.status = {}
          end
          if desiredObj.generation == nil then
            desiredObj.generation = 0
          end
          generation = desiredObj.generation
          replicas = 0
          updatedReplicas = 0 
          readyReplicas = 0
          availableReplicas = 0
          updatedReadyReplicas = 0
          expectedUpdatedReplicas = 0
          updateRevision = ''
          currentRevision = ''
          for i = 1, #statusItems do
            if statusItems[i].status ~= nil and statusItems[i].status.replicas ~= nil then
              replicas = replicas + statusItems[i].status.replicas
            end

            if statusItems[i].status ~= nil and statusItems[i].status.availableReplicas ~= nil then
              availableReplicas = availableReplicas + statusItems[i].status.availableReplicas
            end

          end
          desiredObj.status.observedGeneration = generation
          desiredObj.status.replicas = replicas
          desiredObj.status.updatedReplicas = updatedReplicas
          desiredObj.status.readyReplicas = readyReplicas
          desiredObj.status.availableReplicas = availableReplicas
          desiredObj.status.updatedReadyReplicas = updatedReadyReplicas
          desiredObj.status.expectedUpdatedReplicas = expectedUpdatedReplicas
          desiredObj.status.updateRevision = updateRevision
          desiredObj.status.currentRevision = currentRevision
          return desiredObj
        end

krmada control plane:

➤ k get clone -oyaml

[Poor12]

/lgtm

[RainbowMango]

/lgtm Leave approve to @XiShanYongYe-Chang

[karmada-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: XiShanYongYe-Chang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[pkg/resourceinterpreter/OWNERS](https://github.com/karmada-io/karmada/blob/master/pkg/resourceinterpreter/OWNERS)~~ [XiShanYongYe-Chang] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment