Closed Sashwat-K closed 2 months ago
@dmacvicar , please help here
I think this issue is related to libvirt directly more than to this provider. I would advise to try to reproduce this error using virt-install
. If you are able to do so, that's libvirt bug, not terraform-provider-libvirt
one.
Hello @scabala ,
I used virt-install and I am able to bring up an instance.
I used debug mode to see what is in the XML config generated by the provider. I can see it is defining <acpi>
.
<domain type="kvm">
<name>sashwatk_onprem_crypto_passthrough-domain</name>
<memory unit="MiB">8192</memory>
<vcpu>2</vcpu>
<os>
<type machine="s390-ccw-virtio">hvm</type>
</os>
<features>
<pae></pae>
<acpi></acpi>
<apic></apic>
</features>
<cpu></cpu>
<devices>
<disk type="volume" device="disk">
<driver name="qemu" type="qcow2"></driver>
<source pool="sashwatk_onprem_cryptopassthrough_pool" volume="sashwatk_onprem_crypto_passthrough-vda"></source>
<target dev="vda" bus="virtio"></target>
</disk>
<disk type="volume" device="disk">
<driver name="qemu" type="raw"></driver>
<source pool="sashwatk_onprem_cryptopassthrough_pool" volume="sashwatk_onprem_crypto_passthrough-vdc"></source>
<target dev="vdb" bus="virtio"></target>
</disk>
<interface type="network">
<mac address="52:54:00:02:b6:df"></mac>
<source network="default"></source>
<model type="virtio"></model>
</interface>
<console>
<target type="sclp" port="0"></target>
</console>
<channel type="unix">
<target type="virtio" name="org.qemu.guest_agent.0"></target>
</channel>
<rng model="virtio">
<backend model="random">/dev/urandom</backend>
</rng>
</devices>
</domain>
Would you guide me where should I raise the issue?
Thank you for the response.
In that case I think this might be provider issue. Could you share the domain_update.xsl
file?
Hello @scabala , The following the xsl file contents.
<?xml version="1.0" ?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="yes" indent="yes"/>
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*"/>
</xsl:copy>
</xsl:template>
<xsl:template match="/domain/devices/disk/driver">
<xsl:copy>
<xsl:attribute name="iommu">on</xsl:attribute>
<xsl:apply-templates select="node()|@*" />
</xsl:copy>
</xsl:template>
<xsl:template match="/domain/devices">
<xsl:copy>
<xsl:apply-templates select="@*|node()"/>
<memballoon model="none"/>
</xsl:copy>
</xsl:template>
<xsl:variable name="i" select="position()" />
<xsl:template match="/domain/devices/interface">
<xsl:copy>
<xsl:apply-templates select="@*|node()"/>
<driver name="vhost" iommu="on"/>
</xsl:copy>
</xsl:template>
<xsl:template match="/domain/devices/controller[@type='virtio-serial']"/>
<xsl:template match="/domain/devices/channel"/>
<xsl:template match="/domain/devices/audio"/>
<xsl:template match="/domain/devices/rng"/>
</xsl:stylesheet>
Hello @scabala , Did you get a chance to take a look into this issue?
Hi, I think this is caused by provider adding fields to VM definition but I haven't checked it - from you provided it looks like xsl file does not mess with ACPI so that's the only logical conclusion.
Hello @dmacvicar , Could you please help here?
@Sashwat-K I could just remove ACPI on s390, but I am not sure if I will break something for someone else. How can I know for sure?
Can you attach the output of
virsh capabilities
and
virsh domcapabilities
against the host?
Related to #746
Can you attach the output of
virsh capabilities
and
virsh domcapabilities
against the host?
Hello @dmacvicar
hpvs@dal3-qz2-sr3-rk133-a07:~$ virsh capabilities
<capabilities>
<host>
<uuid>6bbece2a-53b1-42f6-bb9c-850302cb83e2</uuid>
<cpu>
<arch>s390x</arch>
<topology sockets='1' dies='1' cores='8' threads='1'/>
<pages unit='KiB' size='4'/>
<pages unit='KiB' size='1024'/>
</cpu>
<power_management/>
<iommu support='yes'/>
<migration_features>
<live/>
<uri_transports>
<uri_transport>tcp</uri_transport>
<uri_transport>rdma</uri_transport>
</uri_transports>
</migration_features>
<topology>
<cells num='1'>
<cell id='0'>
<memory unit='KiB'>51280124</memory>
<pages unit='KiB' size='4'>12820031</pages>
<pages unit='KiB' size='1024'>0</pages>
<distances>
<sibling id='0' value='10'/>
</distances>
<cpus num='8'>
<cpu id='0' socket_id='1' die_id='0' core_id='0' siblings='0-1'/>
<cpu id='1' socket_id='1' die_id='0' core_id='0' siblings='0-1'/>
<cpu id='2' socket_id='1' die_id='0' core_id='1' siblings='2-3'/>
<cpu id='3' socket_id='1' die_id='0' core_id='1' siblings='2-3'/>
<cpu id='4' socket_id='1' die_id='0' core_id='2' siblings='4-5'/>
<cpu id='5' socket_id='1' die_id='0' core_id='2' siblings='4-5'/>
<cpu id='6' socket_id='1' die_id='0' core_id='3' siblings='6-7'/>
<cpu id='7' socket_id='1' die_id='0' core_id='3' siblings='6-7'/>
</cpus>
</cell>
</cells>
</topology>
<secmodel>
<model>apparmor</model>
<doi>0</doi>
</secmodel>
<secmodel>
<model>dac</model>
<doi>0</doi>
<baselabel type='kvm'>+64055:+994</baselabel>
<baselabel type='qemu'>+64055:+994</baselabel>
</secmodel>
</host>
<guest>
<os_type>hvm</os_type>
<arch name='s390x'>
<wordsize>64</wordsize>
<emulator>/usr/bin/qemu-system-s390x</emulator>
<machine maxCpus='248'>s390-ccw-virtio-noble</machine>
<machine canonical='s390-ccw-virtio-noble' maxCpus='248'>s390-ccw-virtio</machine>
<machine maxCpus='248'>s390-ccw-virtio-disco</machine>
<machine maxCpus='248'>s390-ccw-virtio-8.0</machine>
<machine maxCpus='248'>s390-ccw-virtio-6.0</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.11</machine>
<machine maxCpus='248'>s390-ccw-virtio-7.0</machine>
<machine maxCpus='248'>s390-ccw-virtio-mantic</machine>
<machine maxCpus='248'>s390-ccw-virtio-xenial</machine>
<machine maxCpus='248'>s390-ccw-virtio-5.0</machine>
<machine maxCpus='248'>s390-ccw-virtio-focal</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
<machine maxCpus='248'>s390-ccw-virtio-4.0</machine>
<machine maxCpus='248'>s390-ccw-virtio-zesty</machine>
<machine maxCpus='248'>s390-ccw-virtio-8.2</machine>
<machine maxCpus='248'>s390-ccw-virtio-6.2</machine>
<machine maxCpus='248'>s390-ccw-virtio-3.0</machine>
<machine maxCpus='248'>s390-ccw-virtio-groovy</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
<machine maxCpus='248'>s390-ccw-virtio-7.2</machine>
<machine maxCpus='1'>x-remote</machine>
<machine maxCpus='248'>s390-ccw-virtio-5.2</machine>
<machine maxCpus='248'>s390-ccw-virtio-artful</machine>
<machine maxCpus='248'>s390-ccw-virtio-hirsute</machine>
<machine maxCpus='248'>s390-ccw-virtio-4.2</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.10</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
<machine maxCpus='248'>s390-ccw-virtio-lunar</machine>
<machine maxCpus='248'>s390-ccw-virtio-eoan</machine>
<machine maxCpus='248'>s390-ccw-virtio-kinetic</machine>
<machine maxCpus='248'>s390-ccw-virtio-8.1</machine>
<machine maxCpus='248'>s390-ccw-virtio-yakkety</machine>
<machine maxCpus='248'>s390-ccw-virtio-6.1</machine>
<machine maxCpus='248'>s390-ccw-virtio-cosmic</machine>
<machine maxCpus='248'>s390-ccw-virtio-bionic</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.12</machine>
<machine maxCpus='248'>s390-ccw-virtio-7.1</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
<machine maxCpus='248'>s390-ccw-virtio-5.1</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.9</machine>
<machine maxCpus='248'>s390-ccw-virtio-impish</machine>
<machine maxCpus='248'>s390-ccw-virtio-4.1</machine>
<machine maxCpus='248'>s390-ccw-virtio-jammy</machine>
<machine maxCpus='248'>s390-ccw-virtio-3.1</machine>
<machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
<domain type='qemu'/>
<domain type='kvm'/>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<disksnapshot default='on' toggle='no'/>
<externalSnapshot/>
</features>
</guest>
</capabilities>
hpvs@dal3-qz2-sr3-rk133-a07:~$ virsh domcapabilities
<domainCapabilities>
<path>/usr/bin/qemu-system-s390x</path>
<domain>kvm</domain>
<machine>s390-ccw-virtio-noble</machine>
<arch>s390x</arch>
<vcpu max='248'/>
<iothreads supported='yes'/>
<os supported='yes'>
<enum name='firmware'/>
<loader supported='yes'>
<enum name='type'>
<value>rom</value>
<value>pflash</value>
</enum>
<enum name='readonly'>
<value>yes</value>
<value>no</value>
</enum>
<enum name='secure'>
<value>no</value>
</enum>
</loader>
</os>
<cpu>
<mode name='host-passthrough' supported='yes'>
<enum name='hostPassthroughMigratable'>
<value>off</value>
</enum>
</mode>
<mode name='maximum' supported='yes'>
<enum name='maximumMigratable'>
<value>on</value>
<value>off</value>
</enum>
</mode>
<mode name='host-model' supported='yes'>
<model fallback='forbid'>gen16b-base</model>
<feature policy='require' name='nnpa'/>
<feature policy='require' name='aen'/>
<feature policy='require' name='cmmnt'/>
<feature policy='require' name='vxpdeh'/>
<feature policy='require' name='aefsi'/>
<feature policy='require' name='diag318'/>
<feature policy='require' name='csske'/>
<feature policy='require' name='mepoch'/>
<feature policy='require' name='msa9'/>
<feature policy='require' name='msa8'/>
<feature policy='require' name='msa7'/>
<feature policy='require' name='msa6'/>
<feature policy='require' name='msa5'/>
<feature policy='require' name='msa4'/>
<feature policy='require' name='msa3'/>
<feature policy='require' name='msa2'/>
<feature policy='require' name='msa1'/>
<feature policy='require' name='sthyi'/>
<feature policy='require' name='edat'/>
<feature policy='require' name='ri'/>
<feature policy='require' name='deflate'/>
<feature policy='require' name='edat2'/>
<feature policy='require' name='unpack'/>
<feature policy='require' name='etoken'/>
<feature policy='require' name='vx'/>
<feature policy='require' name='ipter'/>
<feature policy='require' name='pai'/>
<feature policy='require' name='paie'/>
<feature policy='require' name='mepochptff'/>
<feature policy='require' name='ap'/>
<feature policy='require' name='vxeh'/>
<feature policy='require' name='vxpd'/>
<feature policy='require' name='esop'/>
<feature policy='require' name='msa9_pckmo'/>
<feature policy='require' name='vxeh2'/>
<feature policy='require' name='esort'/>
<feature policy='require' name='appv'/>
<feature policy='require' name='apqi'/>
<feature policy='require' name='apft'/>
<feature policy='require' name='els'/>
<feature policy='require' name='iep'/>
<feature policy='require' name='appvi'/>
<feature policy='require' name='apqci'/>
<feature policy='require' name='cte'/>
<feature policy='require' name='ais'/>
<feature policy='require' name='bpb'/>
<feature policy='require' name='ctop'/>
<feature policy='require' name='gs'/>
<feature policy='require' name='ppa15'/>
<feature policy='require' name='zpci'/>
<feature policy='require' name='rdp'/>
<feature policy='require' name='sea_esop2'/>
<feature policy='require' name='beareh'/>
<feature policy='require' name='te'/>
<feature policy='require' name='cmm'/>
<feature policy='require' name='vxpdeh2'/>
</mode>
<mode name='custom' supported='yes'>
<model usable='yes' vendor='IBM'>z13</model>
<model usable='yes' vendor='IBM'>z990.3</model>
<model usable='yes' vendor='IBM'>z13-base</model>
<model usable='yes' vendor='IBM'>z990.2</model>
<model usable='yes' vendor='IBM'>z196.2-base</model>
<model usable='yes' vendor='IBM'>z14</model>
<model usable='yes' vendor='IBM'>z14.2-base</model>
<model usable='yes' vendor='IBM'>z990.4</model>
<model usable='yes' vendor='IBM'>z196</model>
<model usable='yes' vendor='IBM'>z10BC.2-base</model>
<model usable='yes' vendor='IBM'>z114-base</model>
<model usable='yes' vendor='IBM'>gen16b</model>
<model usable='yes' vendor='IBM'>z890.3-base</model>
<model usable='yes' vendor='IBM'>gen16a</model>
<model usable='yes' vendor='IBM'>z990.5</model>
<model usable='yes' vendor='IBM'>z196.2</model>
<model usable='yes' vendor='IBM'>zEC12</model>
<model usable='yes' vendor='IBM'>zEC12.2</model>
<model usable='yes' vendor='IBM'>z9EC.2</model>
<model usable='yes' vendor='IBM'>z9BC</model>
<model usable='yes' vendor='IBM'>z9EC.3-base</model>
<model usable='yes' vendor='IBM'>z800-base</model>
<model usable='yes' vendor='IBM'>z14ZR1-base</model>
<model usable='yes' vendor='IBM'>z890</model>
<model usable='yes' vendor='IBM'>z990.4-base</model>
<model usable='yes' vendor='IBM'>z9EC.3</model>
<model usable='yes' vendor='IBM'>z13.2</model>
<model usable='yes' vendor='IBM'>z10BC.2</model>
<model usable='yes' vendor='IBM'>gen15a-base</model>
<model usable='yes' vendor='IBM'>z14.2</model>
<model usable='yes' vendor='IBM'>zBC12-base</model>
<model usable='yes' vendor='IBM'>gen16a-base</model>
<model usable='yes' vendor='IBM'>z900.3</model>
<model usable='yes' vendor='IBM'>z196-base</model>
<model usable='yes' vendor='IBM'>z990</model>
<model usable='yes' vendor='IBM'>z14-base</model>
<model usable='yes' vendor='IBM'>z900.2</model>
<model usable='yes' vendor='IBM'>z800</model>
<model usable='yes' vendor='IBM'>z10EC</model>
<model usable='yes' vendor='IBM'>z14ZR1</model>
<model usable='yes' vendor='IBM'>z10EC.2</model>
<model usable='yes' vendor='IBM'>z900</model>
<model usable='yes' vendor='IBM'>z10EC.2-base</model>
<model usable='yes' vendor='IBM'>z900.2-base</model>
<model usable='yes' vendor='IBM'>z900-base</model>
<model usable='yes' vendor='IBM'>z10EC.3</model>
<model usable='yes' vendor='IBM'>z990.5-base</model>
<model usable='yes' vendor='unknown'>max</model>
<model usable='yes' vendor='IBM'>gen15b</model>
<model usable='yes' vendor='IBM'>z9BC.2-base</model>
<model usable='yes' vendor='IBM'>gen15a</model>
<model usable='yes' vendor='IBM'>z990.2-base</model>
<model usable='yes' vendor='IBM'>z10EC-base</model>
<model usable='yes' vendor='IBM'>gen16b-base</model>
<model usable='yes' vendor='IBM'>z114</model>
<model usable='yes' vendor='IBM'>z9EC-base</model>
<model usable='yes' vendor='IBM'>z13s</model>
<model usable='yes' vendor='unknown'>qemu</model>
<model usable='yes' vendor='IBM'>gen15b-base</model>
<model usable='yes' vendor='IBM'>zEC12-base</model>
<model usable='yes' vendor='IBM'>z890.3</model>
<model usable='yes' vendor='IBM'>z10EC.3-base</model>
<model usable='yes' vendor='IBM'>z890.2-base</model>
<model usable='yes' vendor='IBM'>z890.2</model>
<model usable='yes' vendor='IBM'>zEC12.2-base</model>
<model usable='yes' vendor='IBM'>z10BC-base</model>
<model usable='yes' vendor='IBM'>z890-base</model>
<model usable='yes' vendor='IBM'>z9BC-base</model>
<model usable='yes' vendor='IBM'>z10BC</model>
<model usable='yes' vendor='IBM'>z9EC</model>
<model usable='yes' vendor='IBM'>z9EC.2-base</model>
<model usable='yes' vendor='IBM'>z13.2-base</model>
<model usable='yes' vendor='IBM'>z900.3-base</model>
<model usable='yes' vendor='IBM'>z990.3-base</model>
<model usable='yes' vendor='IBM'>z13s-base</model>
<model usable='yes' vendor='IBM'>zBC12</model>
<model usable='yes' vendor='IBM'>z990-base</model>
<model usable='yes' vendor='IBM'>z9BC.2</model>
</mode>
</cpu>
<memoryBacking supported='yes'>
<enum name='sourceType'>
<value>file</value>
<value>anonymous</value>
<value>memfd</value>
</enum>
</memoryBacking>
<devices>
<disk supported='yes'>
<enum name='diskDevice'>
<value>disk</value>
<value>cdrom</value>
<value>floppy</value>
<value>lun</value>
</enum>
<enum name='bus'>
<value>fdc</value>
<value>scsi</value>
<value>virtio</value>
<value>usb</value>
</enum>
<enum name='model'>
<value>virtio</value>
<value>virtio-transitional</value>
<value>virtio-non-transitional</value>
</enum>
</disk>
<graphics supported='yes'>
<enum name='type'>
<value>sdl</value>
<value>vnc</value>
<value>egl-headless</value>
<value>dbus</value>
</enum>
</graphics>
<video supported='yes'>
<enum name='modelType'>
<value>virtio</value>
<value>none</value>
</enum>
</video>
<hostdev supported='yes'>
<enum name='mode'>
<value>subsystem</value>
</enum>
<enum name='startupPolicy'>
<value>default</value>
<value>mandatory</value>
<value>requisite</value>
<value>optional</value>
</enum>
<enum name='subsysType'>
<value>usb</value>
<value>pci</value>
<value>scsi</value>
</enum>
<enum name='capsType'/>
<enum name='pciBackend'>
<value>default</value>
<value>vfio</value>
</enum>
</hostdev>
<rng supported='yes'>
<enum name='model'>
<value>virtio</value>
<value>virtio-transitional</value>
<value>virtio-non-transitional</value>
</enum>
<enum name='backendModel'>
<value>random</value>
<value>egd</value>
<value>builtin</value>
</enum>
</rng>
<filesystem supported='yes'>
<enum name='driverType'>
<value>path</value>
<value>handle</value>
<value>virtiofs</value>
</enum>
</filesystem>
<tpm supported='no'/>
<redirdev supported='yes'>
<enum name='bus'>
<value>usb</value>
</enum>
</redirdev>
<channel supported='yes'>
<enum name='type'>
<value>pty</value>
<value>unix</value>
</enum>
</channel>
<crypto supported='yes'>
<enum name='model'>
<value>virtio</value>
</enum>
<enum name='type'>
<value>qemu</value>
</enum>
<enum name='backendModel'>
<value>builtin</value>
</enum>
</crypto>
</devices>
<features>
<gic supported='no'/>
<vmcoreinfo supported='no'/>
<genid supported='no'/>
<backingStoreInput supported='yes'/>
<backup supported='yes'/>
<async-teardown supported='yes'/>
<s390-pv supported='yes'/>
<sev supported='no'/>
<sgx supported='no'/>
</features>
</domainCapabilities>
hpvs@dal3-qz2-sr3-rk133-a07:~$
@Sashwat-K I could just remove ACPI on s390, but I am not sure if I will break something for someone else. How can I know for sure?
- Why not just using the XSL feature and remove the ACPI element?
- Otherwise, could IBM sponsor a CI node on s390x to run the acceptance testsuite on it?
Hey @dmacvicar , If I remember correctly, s390x doesn't support ACPI. So, I dont think that will be an issue.
I tried XSL method, the file is present in the comments above. The issue is still there.
We can't provide an image. But, I am ready to actively support you on any s390x related issues.
@Sashwat-K the XSL file you provided does not touch ACPI so there's no possibility for it to remove it, at least that's how I understand it. If you want to remove ACPI feature using XSL, do it.
I am not sure on how to remove ACPI. Would you help?
Sashwat, sorry, with all my respect :pray: we are in the same place, and I would also need to google/remember the right XSL to remove an element from XML file. Just google/ChatGPT and make it remove the <acpi>
element.
I also opened https://github.com/dmacvicar/terraform-provider-libvirt/discussions as a forum for help and sharing of recipes, though is quite new.
System Information
Linux distribution
Ubuntu 24.04
Terraform version
Provider and libvirt versions
Description of Issue/Question
Setup
Steps to Reproduce Issue
Error
Additional information:
This terraform script is working on Ubuntu 22.04 and RHEL 8.4. But, not working in Ubuntu 24.04
I tried manually creating using
virsh define
andvirt-install
and it is working.