qcow2 files for OCP node storage leads to poor CPU performance

IBM / Ansible-OpenShift-Provisioning

Automate the deployment of Red Hat OpenShift Container Platform on IBM zSystems (s390x). Automated User-Provisoned Infrastructure (UPI) setup using Kernel-based Virtual Machine (KVM).

https://ibm.github.io/Ansible-OpenShift-Provisioning/

MIT License

20 stars 42 forks source link

qcow2 files for OCP node storage leads to poor CPU performance #217

Open vmorris opened 11 months ago

vmorris commented 11 months ago

We've observed very high iowait time being consumed by OCP cluster nodes when they are performing high amount of disk IO.

Please update the automation to give the option to use raw block (SCSI LUN or DASD) devices and pass these through to the OCP guest nodes.

AmadeusPodvratnik commented 11 months ago

More information about implementation can be found here: https://www.redbooks.ibm.com/redbooks/pdfs/sg248463.pdf Chapter 3.6

RH Bug: https://issues.redhat.com/browse/ITIBM-215

Info: https://www.cnblogs.com/popsuper1982/p/3845437.html

vmorris commented 11 months ago

Another team uses the unsupported IPI method to deploy their clusters, and are using qcow2. I wonder if applying the pattern in this tuning playbook would be sufficient to avoid the iowait issue, and continue to use qcow2?

https://github.com/ibm-s390-cloud/ocp-kvm-ipi-automation/blob/main/ansible/roles/tuning/tasks/libvirt_optimize_disks.yml

vmorris commented 11 months ago

Are there tuning playbooks for these optimizations? https://docs.openshift.com/container-platform/4.13/scalability_and_performance/ibm-z-recommended-host-practices.html