Kubevirt VM fails to create DV when creating VM from OCP Virt catalog

[jangel97]

I am trying to create a template from openshift 4.13

This is how my template looks:

apiVersion: template.openshift.io/v1
kind: Template
metadata:
  annotations:
    defaults.template.kubevirt.io/disk: rootdisk
    description: RHIT RHEL9 Medium Server
    iconClass: icon-rhel
    name.os.template.kubevirt.io/rhel9.0: Red Hat Enterprise Linux 9.0 or higher
    name.os.template.kubevirt.io/rhel9.1: Red Hat Enterprise Linux 9.0 or higher
    openshift.io/display-name: Red Hat IT RHEL9
    openshift.io/documentation-url: https://github.com/kubevirt/common-templates # TODO change and point to our docs
    openshift.io/provider-display-name: Skunk
    openshift.io/support-url: https://github.com/kubevirt/common-templates/issues # TODO change and point to our docs
    operator-sdk/primary-resource: openshift-cnv/ssp-kubevirt-hyperconverged
    operator-sdk/primary-resource-type: SSP.ssp.kubevirt.io
    tags: hidden,kubevirt,virtualmachine,linux,rhel
    template.kubevirt.io/containerdisks: |
      images.paas/jmorenas/rhel9
    template.kubevirt.io/editable: |
      /objects[0].spec.template.spec.domain.cpu.sockets
      /objects[0].spec.template.spec.domain.cpu.cores
      /objects[0].spec.template.spec.domain.cpu.threads
      /objects[0].spec.template.spec.domain.resources.requests.memory
      /objects[0].spec.template.spec.domain.devices.disks
      /objects[0].spec.template.spec.volumes
      /objects[0].spec.template.spec.networks
    template.kubevirt.io/provider: Skunk
    template.kubevirt.io/provider-support-level: Full
    template.kubevirt.io/provider-url: https://www.redhat.com
    template.kubevirt.io/version: v1alpha1
    template.openshift.io/bindable: "false"
  labels:
    app.kubernetes.io/component: templating
    app.kubernetes.io/managed-by: ssp-operator
    app.kubernetes.io/name: custom-templates
    app.kubernetes.io/part-of: hyperconverged-cluster
    app.kubernetes.io/version: 4.12.0 # check cluster version
    flavor.template.kubevirt.io/medium: "true"
    os.template.kubevirt.io/rhel9.0: "true"
    os.template.kubevirt.io/rhel9.1: "true"
    template.kubevirt.io/type: vm
    template.kubevirt.io/version: v0.24.1
    vm.kubevirt.io/template: dis-rhit-rhel9-medium
    vm.kubevirt.io/template.namespace: openshift-virtualization-os-images
    workload.template.kubevirt.io/server: "true"
  name: dis-rhit-rhel9-medium
  namespace: openshift-virtualization-os-images
objects:
- apiVersion: kubevirt.io/v1
  kind: VirtualMachine
  metadata:
    labels:
      app_code: "${APP_CODE}"
      managed: "DIS"
      servicephase: "Production"
      name: "${NAME}"
      kubevirt.io/domain: "${NAME}"
      kubevirt.io: redhatit
      app: "${NAME}"
    name: "${NAME}"
    annotations:
      redhatit_kubevirtvm_automation_data: "${AUTOMATION_CONFIGMAP}"
  spec:
    dataVolumeTemplates:
    - apiVersion: cdi.kubevirt.io/v1beta1
      kind: DataVolume
      metadata:
        name: "rootdisk-${NAME}"
      spec:
        source:
          registry:
            url: "docker://images.paas/jmorenas/rhel9"
        pvc:
          accessModes:
            - ReadWriteMany
          resources:
            requests:
              storage: "60Gi"
          storageClassName: "netapp-storage-iscsi"
          volumeMode: Block
    running: true
    template:
      metadata:
        labels:
          kubevirt.io/domain: "${NAME}"
      spec:
        domain:
          cpu:
            cores: 1
            sockets: 1
            threads: 1
          devices:
            disks:
              - disk:
                  bus: virtio
                name: rootdisk
              - disk:
                  bus: virtio
                name: cloudinitdisk
            interfaces:
              - masquerade: {}
                name: default
            rng: {}
          features:
            smm:
              enabled: true
          firmware:
            bootloader:
              efi: {}
          resources:
            requests:
              memory: 4Gi
        evictionStrategy: LiveMigrate
        networks:
          - name: default
            pod: {}
        volumes:
          - dataVolume:
              name: "rootdisk-${NAME}"
            name: rootdisk
          - cloudInitNoCloud:
              userData: |
                #cloud-config
                preserve_hostname: true
                users:
                  - name: cloud-user
                    lock_passwd: true
                    ssh_authorized_keys:
                      - ${AUTHORIZED_SSH_PUBLIC_KEY}
            name: cloudinitdisk
parameters:
- name: NAME
  description: The name of the virtual machine.
  required: true
- name: APP_CODE
  description: The app code label for the VM.
  required: true
- name: AUTOMATION_CONFIGMAP
  description: The provisioning configmap for the VM.
  required: true
- name: AUTHORIZED_SSH_PUBLIC_KEY
  description: The authorized public ssh key.
  required: true

Right after creating my template, I try to create a new VM from the catalog. The VM looks like stopped and when I describe the DV I see the following error:

jmorenas-mac:vm-templates jmorenas$ oc describe dv 
Name:        test
Namespace:    poc-serverless-vms
Labels:       kubevirt.io/created-by=4b607fd8-4137-4c43-b05e-877d4cfa4750
Annotations:  cdi.kubevirt.io/storage.deleteAfterCompletion: true
API Version:  cdi.kubevirt.io/v1beta1
Kind:         DataVolume
Metadata:
  Creation Timestamp:  2023-07-11T21:50:13Z
  Generation:          1
  Managed Fields:
    API Version:  cdi.kubevirt.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .:
          f:kubevirt.io/created-by:
        f:ownerReferences:
          .:
          k:{"uid":"4b607fd8-4137-4c43-b05e-877d4cfa4750"}:
      f:spec:
        .:
        f:source:
          .:
          f:registry:
            .:
            f:url:
        f:storage:
          .:
          f:resources:
            .:
            f:requests:
              .:
              f:storage:
      f:status:
    Manager:    virt-controller
    Operation:  Update
    Time:       2023-07-11T21:50:13Z
  Owner References:
    API Version:           kubevirt.io/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  VirtualMachine
    Name:                  test
    UID:                   4b607fd8-4137-4c43-b05e-877d4cfa4750
  Resource Version:        949501584
  UID:                     2ff318de-cda1-481b-a5e8-57a835b0ed82
Spec:
  Source:
    Registry:
      URL:  docker://images.paasjmorenas/rhel9
  Storage:
    Resources:
      Requests:
        Storage:  30Gi
Status:
Events:
  Type     Reason            Age               From                   Message
  ----     ------            ----              ----                   -------
  Warning  ErrClaimNotValid  1s (x11 over 6s)  datavolume-controller  DataVolume.storage spec is missing accessMode and no storageClass to choose profile

I can see a PVC is created but it has incomplete information:

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  annotations:
    cdi.kubevirt.io/storage.deleteAfterCompletion: "true"
  creationTimestamp: "2023-07-11T21:56:13Z"
  generation: 1
  labels:
    kubevirt.io/created-by: 4cdaa161-664b-4140-b26f-4e0eaa3cc9a8
  name: rootdisk-test
  namespace: poc-serverless-vms
  ownerReferences:
  - apiVersion: kubevirt.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: VirtualMachine
    name: test
    uid: 4cdaa161-664b-4140-b26f-4e0eaa3cc9a8
  resourceVersion: "949513908"
  uid: bdb1ec4f-b5fc-471d-9e3a-7e0d4d701189
spec:
  source:
    registry:
      url: docker://images.paas/jmorenas/rhel9
  storage:
    resources:
      requests:
        storage: 30Gi
status: {}

Am I doing something wrong in the template? Is there anything I am missing? The expected behaviour would be that right after creating the VM a DV poinitng to containerdisk docker://images.paas/jmorenas/rhel9 should start downloading the containerdisk.

Thanks in advance,

[awels]

The problem is that your storage profile is incomplete DataVolume.storage spec is missing accessMode and no storageClass to choose profile which usually happens when we have no pre-built storage profile. Since you are using Open Shift, if you run oc get storageprofile you should get a list of storage profile. The name will match the storage class you are using. You have to edit the storage profile and give it appropriate accessMode and volumeMode for your particular storage. Documentation can be found here https://github.com/kubevirt/containerized-data-importer/blob/main/doc/storageprofile.md or https://docs.openshift.com/container-platform/4.13/virt/virtual_machines/virtual_disks/virt-creating-data-volumes.html#virt-customizing-storage-profile_virt-creating-data-volumes

[jangel97]

Hey @awels thank you for your comment!

It is a bit strange because in my StorageProfile I can see this:

apiVersion: cdi.kubevirt.io/v1beta1
kind: StorageProfile
metadata:
  creationTimestamp: "2023-02-13T12:30:16Z"
  generation: 2
  labels:
    app: containerized-data-importer
    app.kubernetes.io/component: storage
    app.kubernetes.io/managed-by: cdi-controller
    app.kubernetes.io/part-of: hyperconverged-cluster
    app.kubernetes.io/version: 4.12.0
    cdi.kubevirt.io: ""
  name: netapp-storage-iscsi
  ownerReferences:
  - apiVersion: cdi.kubevirt.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: CDI
    name: cdi-kubevirt-hyperconverged
    uid: 325181a5-06ec-43d3-ad0b-5ae53e793d76
  resourceVersion: "950783059"
  uid: cf02aaa9-2f86-4f03-a584-c5355bbf7f36
spec:
  claimPropertySets:
  - accessModes:
    - ReadWriteOnce
    volumeMode: Block
status:
  claimPropertySets:
  - accessModes:
    - ReadWriteOnce
    volumeMode: Block
  provisioner: csi.trident.netapp.io
  storageClass: netapp-storage-iscsi

And this is my storageclass netapp-storage-iscsi:

allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: "2023-02-13T12:30:16Z"
  name: netapp-storage-iscsi
  resourceVersion: "658168888"
  uid: fa74ed85-3cd7-49c4-84f8-718c2ae09c05
parameters:
  backendType: ontap-san
  fsType: ext4
  selector: classification=restricted
provisioner: csi.trident.netapp.io
reclaimPolicy: Delete
volumeBindingMode: Immediate

So, according to the docs everything should be set right?

Kind regards,

[awels]

Can you make that storage class the default storage class? Or specify the storage class in the datavolume storage section.

[kubevirt-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[kubevirt-bot]

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[kubevirt-bot]

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

/close

[kubevirt-bot]

@kubevirt-bot: Closing this issue.

In response to [this](https://github.com/kubevirt/kubevirt/issues/10084#issuecomment-1848397683): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.