openshift_process | 'openshift' in template output is replaced by '********' causing subsequent apply to fail

kubealex commented 12 months ago

Hi team,

I was using the openshift_process module, but it appears that for some reasons the result of the rendered template is alterated masking the word 'openshift' with asterisks. Of course using the output of the module to further apply the template fails.

Example (will omit non-related lines)

Output from oc process ns//template: [...] "spec": { "sourceRef": { "kind": "DataSource", "name": "rhel9", "namespace": "openshift-virtualization-os-images" }, [...]

Output from openshift_process module [...] "spec": { "sourceRef": { "kind": "DataSource", "name": "rhel9", "namespace": "****-virtualization-os-images" }, [...]

Of course, using the k8s module later using the output of the module fails because the ****virtualization-os-images namespace does not exist.

The workaround (super ugly) is to use a "| replace('****', 'openshift')" on the output.

I tried to dig where this happens, but I was not able to find any clue.

openshift-bot commented 9 months ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

gravesm commented 9 months ago

/remove-lifecycle stale

openshift-bot commented 5 months ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

gravesm commented 5 months ago

/lifecycle frozen

abikouo commented 4 months ago

@kubealex could you please provide the full details (the template definition and the playbook executed) to reproduce the issue ?

kubealex commented 3 months ago

To replicate it, I tried with the default rhel9-server-small in the openshift namespace. Since the template has "openshift-virtualization-os-image" as the data source namespace, in the "process" module output it becomes "****-virtualization-os-image"

Template

kind: Template
apiVersion: template.openshift.io/v1
metadata:
  name: rhel9-server-small
  namespace: openshift
  uid: 4330ae43-c70d-4333-9092-d79c0151bc88
  resourceVersion: '115879'
  creationTimestamp: '2024-03-06T15:45:18Z'
  labels:
    app.kubernetes.io/part-of: hyperconverged-cluster
    flavor.template.kubevirt.io/small: 'true'
    template.kubevirt.io/version: v0.27.0
    app.kubernetes.io/version: 4.15.0
    template.kubevirt.io/type: base
    app.kubernetes.io/component: templating
    app.kubernetes.io/managed-by: ssp-operator
    os.template.kubevirt.io/rhel9.0: 'true'
    os.template.kubevirt.io/rhel9.1: 'true'
    template.kubevirt.io/default-os-variant: 'true'
    app.kubernetes.io/name: common-templates
    os.template.kubevirt.io/rhel9.2: 'true'
    os.template.kubevirt.io/rhel9.3: 'true'
    os.template.kubevirt.io/rhel9.4: 'true'
    workload.template.kubevirt.io/server: 'true'
  annotations:
    template.kubevirt.io/provider: Red Hat
    template.kubevirt.io/provider-url: 'https://www.redhat.com'
    template.kubevirt.io/containerdisks: |
      registry.redhat.io/rhel9/rhel-guest-image
    template.kubevirt.io/version: v1alpha1
    openshift.io/display-name: Red Hat Enterprise Linux 9 VM
    openshift.io/documentation-url: 'https://github.com/kubevirt/common-templates'
    template.kubevirt.io/images: |
      https://access.redhat.com/downloads/content/479/ver=/rhel---9/9.0/x86_64/product-software
    operator-sdk/primary-resource-type: SSP.ssp.kubevirt.io
    defaults.template.kubevirt.io/disk: rootdisk
    name.os.template.kubevirt.io/rhel9.0: Red Hat Enterprise Linux 9.0 or higher
    name.os.template.kubevirt.io/rhel9.1: Red Hat Enterprise Linux 9.0 or higher
    template.kubevirt.io/editable: |
      /objects[0].spec.template.spec.domain.cpu.sockets
      /objects[0].spec.template.spec.domain.cpu.cores
      /objects[0].spec.template.spec.domain.cpu.threads
      /objects[0].spec.template.spec.domain.memory.guest
      /objects[0].spec.template.spec.domain.devices.disks
      /objects[0].spec.template.spec.volumes
      /objects[0].spec.template.spec.networks
    name.os.template.kubevirt.io/rhel9.2: Red Hat Enterprise Linux 9.0 or higher
    template.openshift.io/bindable: 'false'
    openshift.kubevirt.io/pronounceable-suffix-for-name-expression: 'true'
    operator-sdk/primary-resource: openshift-cnv/ssp-kubevirt-hyperconverged
    name.os.template.kubevirt.io/rhel9.3: Red Hat Enterprise Linux 9.0 or higher
    name.os.template.kubevirt.io/rhel9.4: Red Hat Enterprise Linux 9.0 or higher
    tags: 'hidden,kubevirt,virtualmachine,linux,rhel'
    template.kubevirt.io/provider-support-level: Full
    description: Template for Red Hat Enterprise Linux 9 VM or newer. A PVC with the RHEL disk image must be available.
    openshift.io/support-url: 'https://github.com/kubevirt/common-templates/issues'
    iconClass: icon-rhel
    openshift.io/provider-display-name: Red Hat
  managedFields:
    - manager: manager
      operation: Update
      apiVersion: template.openshift.io/v1
      time: '2024-03-06T15:45:18Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            'f:template.kubevirt.io/editable': {}
            'f:operator-sdk/primary-resource': {}
            'f:operator-sdk/primary-resource-type': {}
            'f:defaults.template.kubevirt.io/disk': {}
            'f:template.kubevirt.io/provider-support-level': {}
            'f:name.os.template.kubevirt.io/rhel9.0': {}
            'f:description': {}
            'f:name.os.template.kubevirt.io/rhel9.1': {}
            'f:template.openshift.io/bindable': {}
            'f:name.os.template.kubevirt.io/rhel9.2': {}
            'f:name.os.template.kubevirt.io/rhel9.3': {}
            'f:iconClass': {}
            'f:openshift.kubevirt.io/pronounceable-suffix-for-name-expression': {}
            'f:name.os.template.kubevirt.io/rhel9.4': {}
            'f:tags': {}
            .: {}
            'f:template.kubevirt.io/provider': {}
            'f:template.kubevirt.io/provider-url': {}
            'f:template.kubevirt.io/containerdisks': {}
            'f:openshift.io/support-url': {}
            'f:openshift.io/provider-display-name': {}
            'f:template.kubevirt.io/images': {}
            'f:openshift.io/display-name': {}
            'f:template.kubevirt.io/version': {}
            'f:openshift.io/documentation-url': {}
          'f:labels':
            'f:os.template.kubevirt.io/rhel9.0': {}
            'f:os.template.kubevirt.io/rhel9.1': {}
            'f:template.kubevirt.io/default-os-variant': {}
            'f:os.template.kubevirt.io/rhel9.2': {}
            'f:os.template.kubevirt.io/rhel9.3': {}
            'f:app.kubernetes.io/managed-by': {}
            'f:os.template.kubevirt.io/rhel9.4': {}
            'f:app.kubernetes.io/name': {}
            .: {}
            'f:app.kubernetes.io/part-of': {}
            'f:workload.template.kubevirt.io/server': {}
            'f:flavor.template.kubevirt.io/small': {}
            'f:app.kubernetes.io/version': {}
            'f:template.kubevirt.io/type': {}
            'f:template.kubevirt.io/version': {}
            'f:app.kubernetes.io/component': {}
        'f:objects': {}
        'f:parameters': {}
objects:
  - apiVersion: kubevirt.io/v1
    kind: VirtualMachine
    metadata:
      annotations:
        vm.kubevirt.io/validations: |
          [
            {
              "name": "minimal-required-memory",
              "path": "jsonpath::.spec.domain.memory.guest",
              "rule": "integer",
              "message": "This VM requires more memory.",
              "min": 1610612736
            }
          ]
      labels:
        app: '${NAME}'
        kubevirt.io/dynamic-credentials-support: 'true'
        vm.kubevirt.io/template: rhel9-server-small
        vm.kubevirt.io/template.revision: '1'
        vm.kubevirt.io/template.version: v0.27.0
      name: '${NAME}'
    spec:
      dataVolumeTemplates:
        - apiVersion: cdi.kubevirt.io/v1beta1
          kind: DataVolume
          metadata:
            name: '${NAME}'
          spec:
            sourceRef:
              kind: DataSource
              name: '${DATA_SOURCE_NAME}'
              namespace: '${DATA_SOURCE_NAMESPACE}'
            storage:
              resources:
                requests:
                  storage: 30Gi
      running: false
      template:
        metadata:
          annotations:
            vm.kubevirt.io/flavor: small
            vm.kubevirt.io/os: rhel9
            vm.kubevirt.io/workload: server
          labels:
            kubevirt.io/domain: '${NAME}'
            kubevirt.io/size: small
        spec:
          domain:
            cpu:
              cores: 1
              sockets: 1
              threads: 1
            devices:
              disks:
                - disk:
                    bus: virtio
                  name: rootdisk
                - disk:
                    bus: virtio
                  name: cloudinitdisk
              interfaces:
                - masquerade: {}
                  model: virtio
                  name: default
              networkInterfaceMultiqueue: true
              rng: {}
            features:
              smm:
                enabled: true
            firmware:
              bootloader:
                efi: {}
            machine:
              type: pc-q35-rhel9.2.0
            memory:
              guest: 2Gi
          networks:
            - name: default
              pod: {}
          terminationGracePeriodSeconds: 180
          volumes:
            - dataVolume:
                name: '${NAME}'
              name: rootdisk
            - cloudInitNoCloud:
                userData: |-
                  #cloud-config
                  user: cloud-user
                  password: ${CLOUD_USER_PASSWORD}
                  chpasswd: { expire: False }
              name: cloudinitdisk
parameters:
  - name: NAME
    description: VM name
    generate: expression
    from: 'rhel9-[a-z0-9]{16}'
  - name: DATA_SOURCE_NAME
    description: Name of the DataSource to clone
    value: rhel9
  - name: DATA_SOURCE_NAMESPACE
    description: Namespace of the DataSource
    value: openshift-virtualization-os-images
  - name: CLOUD_USER_PASSWORD
    description: Randomized password for the cloud-init user cloud-user
    generate: expression
    from: '[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}'

Playbook:

- name: OCP | Deploy VM from template
  hosts: localhost
  module_defaults:
    group/redhat.openshift.openshift:
      host: "https://api.ocp.ocpdemo.labs:6443/"
      username: "admin"
      password: "openshift"
      validate_certs: "{{ ocp_api_verify_tls | default(false, true) }}"
  vars:
    vm_name: "{{ ocp_vm_name | default('rhel9-vm', true) }}"
    vm_namespace: "{{ ocp_vm_namespace | default('openshift', true) }}"
    vm_template: "{{ ocp_vm_template | default('rhel9-server-small', true) }}"
    vm_cpu_count: "{{ ocp_vm_cpu | default(2, true)}}"
    vm_mem_count: "{{ ocp_vm_mem | default('4Gi', true) }}"
  tasks:
    - name: OCP | Login to OCP cluster
      block:
        - name: OCP | Login into OpenShift Container Platform cluster
          redhat.openshift.openshift_auth:
          register: _openshift_auth_results

        - name: Retrieving OCP api_key
          ansible.builtin.set_fact:
            ocp_api_key: "{{ _openshift_auth_results.openshift_auth.api_key }}"

    - name: OCP | Process VM template
      redhat.openshift.openshift_process:
        api_key: "{{ ocp_api_key }}"
        name: "{{ vm_template }}"
        namespace: "{{ vm_namespace }}"
        parameters:
          NAME: "{{ vm_name }}"
        state: rendered
      register: _rendered_template

    - name: debug
      debug:
        var: _rendered_template

Playbook run output:

PLAYBOOK: test_ocp_template.yaml ***********************************************
1 plays in /home/alerossi/test_ocp_template.yaml

PLAY [OCP | Deploy VM from template] *******************************************

TASK [Gathering Facts] *********************************************************
task path: /home/alerossi/test_ocp_template.yaml:1
ok: [localhost]

TASK [OCP | Login into OpenShift Container Platform cluster] *******************
task path: /home/alerossi/test_ocp_template.yaml:18
ok: [localhost] => {"changed": false, "k8s_auth": {"api_key": "sha256~VA7QB8MZy2laEcgdwLxcfkKmdh694Sc4wKLoLHsX7WU", "ca_cert": null, "host": "https://api.ocp.ocpdemo.labs:6443/", "username": "admin", "validate_certs": false}, "openshift_auth": {"api_key": "sha256~VA7QB8MZy2laEcgdwLxcfkKmdh694Sc4wKLoLHsX7WU", "ca_cert": null, "host": "https://api.ocp.ocpdemo.labs:6443/", "username": "admin", "validate_certs": false}}

TASK [Retrieving OCP api_key] **************************************************
task path: /home/alerossi/test_ocp_template.yaml:22
ok: [localhost] => {"ansible_facts": {"ocp_api_key": "sha256~VA7QB8MZy2laEcgdwLxcfkKmdh694Sc4wKLoLHsX7WU"}, "changed": false}

TASK [OCP | Process VM template] ***********************************************
task path: /home/alerossi/test_ocp_template.yaml:26
ok: [localhost] => {"changed": false, "message": "", "resources": [{"apiVersion": "kubevirt.io/v1", "kind": "VirtualMachine", "metadata": {"annotations": {"vm.kubevirt.io/validations": "[\n  {\n    \"name\": \"minimal-required-memory\",\n    \"path\": \"jsonpath::.spec.domain.memory.guest\",\n    \"rule\": \"integer\",\n    \"message\": \"This VM requires more memory.\",\n    \"min\": 1610612736\n  }\n]\n"}, "labels": {"app": "rhel9-vm", "kubevirt.io/dynamic-credentials-support": "true", "vm.kubevirt.io/template": "rhel9-server-small", "vm.kubevirt.io/template.revision": "1", "vm.kubevirt.io/template.version": "v0.27.0"}, "name": "rhel9-vm"}, "spec": {"dataVolumeTemplates": [{"apiVersion": "cdi.kubevirt.io/v1beta1", "kind": "DataVolume", "metadata": {"name": "rhel9-vm"}, "spec": {"sourceRef": {"kind": "DataSource", "name": "rhel9", "namespace": "********-virtualization-os-images"}, "storage": {"resources": {"requests": {"storage": "30Gi"}}}}}], "running": false, "template": {"metadata": {"annotations": {"vm.kubevirt.io/flavor": "small", "vm.kubevirt.io/os": "rhel9", "vm.kubevirt.io/workload": "server"}, "labels": {"kubevirt.io/domain": "rhel9-vm", "kubevirt.io/size": "small"}}, "spec": {"domain": {"cpu": {"cores": 1, "sockets": 1, "threads": 1}, "devices": {"disks": [{"disk": {"bus": "virtio"}, "name": "rootdisk"}, {"disk": {"bus": "virtio"}, "name": "cloudinitdisk"}], "interfaces": [{"masquerade": {}, "model": "virtio", "name": "default"}], "networkInterfaceMultiqueue": true, "rng": {}}, "features": {"smm": {"enabled": true}}, "firmware": {"bootloader": {"efi": {}}}, "machine": {"type": "pc-q35-rhel9.2.0"}, "memory": {"guest": "2Gi"}}, "networks": [{"name": "default", "pod": {}}], "terminationGracePeriodSeconds": 180, "volumes": [{"dataVolume": {"name": "rhel9-vm"}, "name": "rootdisk"}, {"cloudInitNoCloud": {"userData": "#cloud-config\nuser: cloud-user\npassword: sjxq-u4wa-a6la\nchpasswd: { expire: False }"}, "name": "cloudinitdisk"}]}}}}]}

TASK [debug] *******************************************************************
task path: /home/alerossi/test_ocp_template.yaml:36
ok: [localhost] => {
    "_rendered_template": {
        "changed": false,
        "failed": false,
        "message": "",
        "resources": [
            {
                "apiVersion": "kubevirt.io/v1",
                "kind": "VirtualMachine",
                "metadata": {
                    "annotations": {
                        "vm.kubevirt.io/validations": "[\n  {\n    \"name\": \"minimal-required-memory\",\n    \"path\": \"jsonpath::.spec.domain.memory.guest\",\n    \"rule\": \"integer\",\n    \"message\": \"This VM requires more memory.\",\n    \"min\": 1610612736\n  }\n]\n"
                    },
                    "labels": {
                        "app": "rhel9-vm",
                        "kubevirt.io/dynamic-credentials-support": "true",
                        "vm.kubevirt.io/template": "rhel9-server-small",
                        "vm.kubevirt.io/template.revision": "1",
                        "vm.kubevirt.io/template.version": "v0.27.0"
                    },
                    "name": "rhel9-vm"
                },
                "spec": {
                    "dataVolumeTemplates": [
                        {
                            "apiVersion": "cdi.kubevirt.io/v1beta1",
                            "kind": "DataVolume",
                            "metadata": {
                                "name": "rhel9-vm"
                            },
                            "spec": {
                                "sourceRef": {
                                    "kind": "DataSource",
                                    "name": "rhel9",
                                    "namespace": "********-virtualization-os-images"
                                },
                                "storage": {
                                    "resources": {
                                        "requests": {
                                            "storage": "30Gi"
                                        }
                                    }
                                }
                            }
                        }
                    ],
                    "running": false,
                    "template": {
                        "metadata": {
                            "annotations": {
                                "vm.kubevirt.io/flavor": "small",
                                "vm.kubevirt.io/os": "rhel9",
                                "vm.kubevirt.io/workload": "server"
                            },
                            "labels": {
                                "kubevirt.io/domain": "rhel9-vm",
                                "kubevirt.io/size": "small"
                            }
                        },
                        "spec": {
                            "domain": {
                                "cpu": {
                                    "cores": 1,
                                    "sockets": 1,
                                    "threads": 1
                                },
                                "devices": {
                                    "disks": [
                                        {
                                            "disk": {
                                                "bus": "virtio"
                                            },
                                            "name": "rootdisk"
                                        },
                                        {
                                            "disk": {
                                                "bus": "virtio"
                                            },
                                            "name": "cloudinitdisk"
                                        }
                                    ],
                                    "interfaces": [
                                        {
                                            "masquerade": {},
                                            "model": "virtio",
                                            "name": "default"
                                        }
                                    ],
                                    "networkInterfaceMultiqueue": true,
                                    "rng": {}
                                },
                                "features": {
                                    "smm": {
                                        "enabled": true
                                    }
                                },
                                "firmware": {
                                    "bootloader": {
                                        "efi": {}
                                    }
                                },
                                "machine": {
                                    "type": "pc-q35-rhel9.2.0"
                                },
                                "memory": {
                                    "guest": "2Gi"
                                }
                            },
                            "networks": [
                                {
                                    "name": "default",
                                    "pod": {}
                                }
                            ],
                            "terminationGracePeriodSeconds": 180,
                            "volumes": [
                                {
                                    "dataVolume": {
                                        "name": "rhel9-vm"
                                    },
                                    "name": "rootdisk"
                                },
                                {
                                    "cloudInitNoCloud": {
                                        "userData": "#cloud-config\nuser: cloud-user\npassword: sjxq-u4wa-a6la\nchpasswd: { expire: False }"
                                    },
                                    "name": "cloudinitdisk"
                                }
                            ]
                        }
                    }
                }
            }
        ]
    }
}

PLAY RECAP *********************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

openshift / community.okd

openshift_process | 'openshift' in template output is replaced by '********' causing subsequent apply to fail #198