konveyor / move2kube

Move2Kube is a command-line tool for automating creation of Infrastructure as code (IaC) artifacts. It has inbuilt support for creating IaC artifacts for replatforming to Kubernetes/Openshift.
https://move2kube.konveyor.io/
Apache License 2.0
383 stars 118 forks source link

feat: consolidate all the conditions that trigger a transformer in one place #882

Open HarikrishnanBalagopal opened 1 year ago

HarikrishnanBalagopal commented 1 year ago

Is your feature request related to a problem? Please describe.

Currently we have a bunch of different fields to indicate when a transformer might run.

Examples:

https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/add-custom-files-directories-in-custom-locations/transformer.yaml#L9-L16

  directoryDetect:
    levels: -1
  consumes:
    Service:
      merge: false
  override:
    matchLabels: 
      move2kube.konveyor.io/built-in: "true"

https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/custom-default-transformer/transformer.yaml#L9-L14

  directoryDetect:
    levels: 0
  config:
    starFile: "defaulttransformer.star"
  invokedByDefault:
    enabled: true

Describe the solution you'd like

We can consolidate all the different conditions under a single field similar to Github workflows:

https://github.com/konveyor/move2kube/blob/6ee8f6be3e7f9b9eb0bb1200b468ac95018f244f/.github/workflows/assignproject.yml#L3-L5

For default transformers:

on:
  transform:
    always: true

For normal transformers that consume artifacts of certain types:

on:
  transform:
    artifacts:
      - type: ArtifactType1
        merge: false
      - type: ArtifactType2

We could also trigger on path mappings.

on:
  transform:
    artifacts:
      - type: ArtifactType1
      - type: ArtifactType2
    pathMappings:
      - type: Source
      - type: Template

This could be useful if we want to add a common LICENSE on top of each file we generate.

Running after other transformers:

on:
  transform:
    transformers:
      - runAfter: true
        matchLabels:
          move2kube.konveyor.io/name: CloudFoundry

For transformers that run during both the plan and transform phase:

on:
  plan:
    directories: all
  transform:
    artifacts:
      - type: ArtifactType1
        merge: true
      - type: ArtifactType2
        merge: false

This is also a good chance to move away from 1, 0, -1 for the levels and use more meaningful names.

Additional context

Since we are moving to v0.4 this is a good time to add any breaking changes we would like.

HarikrishnanBalagopal commented 1 year ago

Example of how an existing Helm chart transformer would change: https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/add-custom-files-directories-in-custom-locations/transformer.yaml

Also note the change to apiVersion: move2kube.konveyor.io/v1beta1 so that we can be backwards-compatible and continue to support the older transformer yaml format in v0.4. Or at least detect when we are given the older format and throw a warning/error.

apiVersion: move2kube.konveyor.io/v1alpha1
kind: Transformer
metadata:
  name: CustomHelmChartGen
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Starlark"
  directoryDetect:
    levels: -1
  consumes:
    Service:
      merge: false
  override:
    matchLabels: 
      move2kube.konveyor.io/built-in: "true"
  config:
    starFile: "customhelmchartgen.star"

turns into

apiVersion: move2kube.konveyor.io/v1beta1
kind: Transformer
metadata:
  name: CustomHelmChartGen
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Starlark"
  on:
    plan:
      directories: all
    transform:
      artifacts:
        - type: Service
          merge: false
      transformers:
        - override: true
          matchLabels: 
            move2kube.konveyor.io/built-in: "true"
  config:
    starFile: "customhelmchartgen.star"
HarikrishnanBalagopal commented 1 year ago

Example: https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/cloud-foundry-to-ce-iks-roks/k8s-yamls/iks/transformer.yaml

apiVersion: move2kube.konveyor.io/v1alpha1
kind: Transformer
metadata:
  name: IKSKubernetes
  labels:
    move2kube.konveyor.io/built-in: false
spec:
  class: "Kubernetes"
  directoryDetect:
    levels: 0
  consumes:
    IR:
      merge: true
  produces:
    KubernetesYamls:
      disabled: false
  override:
    matchLabels: 
      move2kube.konveyor.io/name: Kubernetes
  dependency:
    matchLabels:
      move2kube.konveyor.io/name: IKSClusterSelector
  config:
    outputPath: "deploy/yamls/iks"
    ingressName: "{{ .ProjectName }}"

turns into

apiVersion: move2kube.konveyor.io/v1beta1
kind: Transformer
metadata:
  name: IKSKubernetes
  labels:
    move2kube.konveyor.io/built-in: false
spec:
  class: "Kubernetes"
  on:
    transform:
      artifacts:
        - type: IR
      transformers:
        - override: true
          matchLabels: 
            move2kube.konveyor.io/name: Kubernetes
        - runAfter: true
          matchLabels:
            move2kube.konveyor.io/name: IKSClusterSelector
  produces:
    KubernetesYamls:
      disabled: false
  config:
    outputPath: "deploy/yamls/iks"
    ingressName: "{{ .ProjectName }}"

Since we don't want the transformer to run during the plan phase, we can omit that trigger.

HarikrishnanBalagopal commented 1 year ago

Example: https://github.com/konveyor/move2kube-transformers/blob/525d3854a18e9c85582178395099544e3e351363/custom-helm-kustomize-octemplates-parameterization/transformer.yaml

apiVersion: move2kube.konveyor.io/v1alpha1
kind: Transformer
metadata:
  name: AdvancedCustomParameterizer
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Parameterizer"
  directoryDetect:
    levels: -1
  consumes:
    Service:
      merge: false
    KubernetesYamlsInSource:
      merge: false
    KubernetesYamls:
      merge: true
  override:
    matchLabels: 
      move2kube.konveyor.io/name: Parameterizer
  config:
    helmPath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/helm-chart"
    ocTemplatePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/openshift-template"
    kustomizePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/kustomize"
    projectName: "{{ if eq .ArtifactType \"KubernetesYamls\" }}{{ .ProjectName }}{{ else }}{{ if eq .ArtifactType \"KubernetesYamlsInSource\" }}{{ .ArtifactName }}{{ else }}{{ .ServiceName }}{{end}}{{end}}"

turns into

apiVersion: move2kube.konveyor.io/v1beta1
kind: Transformer
metadata:
  name: AdvancedCustomParameterizer
  labels: 
    move2kube.konveyor.io/built-in: false
spec:
  class: "Parameterizer"
  on:
    plan:
      directories: all
    transform:
      artifacts:
        - type: Service
          merge: false
        - type: KubernetesYamlsInSource
          merge: false
        - type: KubernetesYamls
          merge: true
      transformers:
       - override: true
         matchLabels: 
           move2kube.konveyor.io/name: Parameterizer
  config:
    helmPath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/helm-chart"
    ocTemplatePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/openshift-template"
    kustomizePath: "{{ $pathType := EnvPathType .YamlsPath}}{{ $rel := Rel .YamlsPath }}{{ if eq $pathType \"Source\" }}source/{{end}}{{ $rel }}{{ if ne $rel \".\" }}/..{{end}}/{{ FilePathBase .YamlsPath }}-parameterized/kustomize"
    projectName: "{{ if eq .ArtifactType \"KubernetesYamls\" }}{{ .ProjectName }}{{ else }}{{ if eq .ArtifactType \"KubernetesYamlsInSource\" }}{{ .ArtifactName }}{{ else }}{{ .ServiceName }}{{end}}{{end}}"