search

open-power-host-os / qemu

OpenPOWER Host OS qemu repository
Other
2 stars 3 forks source link

Live migration fails with "qemu-kvm: htab_load() bad index in htab stream (htab_shift=0)" #6

Closed balamuruhans closed 7 years ago

balamuruhans commented 7 years ago

Description: Basic Live migration is broken in HostOS 3.0 with "qemu-kvm: htab_load() bad index in htab stream (htab_shift=0)".

Used P9 Boston DD2.0 Host and Fedora 26 Guest:

# cat /proc/cpuinfo | grep revision
revision    : 2.0 (pvr 004e 1200)

Steps to recreate:

  1. Have a healthy guest with qcow2 image with shared storage in NFS
  2. Mount the image location in source and in destination.
  3. Enabled ports 49152:49216 in iptables in destination host, # iptables -I INPUT -p tcp -m tcp --dport 49152:49216 -j ACCEPT
  4. Permit selinux to allow libvirt for using NFS in source and destination, # setsebool virt_use_nfs 1
  5. start the guest in source and perform postcopy migration # virsh start avocado-vt-vm1-migration 
    
# virsh list
Id    Name                           State
----------------------------------------------------
49    avocado-vt-vm1-migration       running
6. Perform migration and issue is observed,

virsh migrate avocado-vt-vm1-migration qemu+ssh://9.40.193.221/system --live --verbose

root@9.40.193.221's password: error: internal error: qemu unexpectedly closed the monitor: 2017-09-28T05:20:14.858662Z qemu-kvm: -chardev pty,id=charserial0: char device redirected to /dev/pts/1 (label charserial0) 2017-09-28T05:20:15.182815Z qemu-kvm: htab_load() bad index 2113929216 (23554+0 entries) in htab stream (htab_shift=0) 2017-09-28T05:20:15.182851Z qemu-kvm: error while loading state for instance 0x0 of device 'spapr/htab' 2017-09-28T05:20:15.184847Z qemu-kvm: load of migration failed: Invalid argument

**Test Environment:** (In source and destination host)
Qemu:

rpm -qa | grep qemu

qemu-img-ev-2.9.0-16.el7_4.5.1.ppc64le ipxe-roms-qemu-20170123-1.git4e85b27.el7_4.1.noarch qemu-kvm-ev-2.9.0-16.el7_4.5.1.ppc64le libvirt-daemon-driver-qemu-3.2.0-14.el7_4.3.ppc64le qemu-kvm-common-ev-2.9.0-16.el7_4.5.1.ppc64le

Libvirt:

rpm -qa | grep libvirt

libvirt-daemon-driver-storage-core-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-config-nwfilter-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-storage-iscsi-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-storage-logical-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-interface-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-network-3.2.0-14.el7_4.3.ppc64le libvirt-client-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-config-network-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-lxc-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-storage-scsi-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-storage-mpath-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-storage-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-secret-3.2.0-14.el7_4.3.ppc64le libvirt-python-3.2.0-3.el7.ppc64le libvirt-devel-3.2.0-14.el7_4.3.ppc64le libvirt-libs-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-nwfilter-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-qemu-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-storage-disk-3.2.0-14.el7_4.3.ppc64le libvirt-daemon-driver-nodedev-3.2.0-14.el7_4.3.ppc64le libvirt-3.2.0-14.el7_4.3.ppc64le

Host kernel:

uname -a

Linux ltc-boston120.aus.stglabs.ibm.com 4.13.0-4.rel.git49564cb.el7.centos.ppc64le #1 SMP Fri Sep 22 22:49:59 -03 2017 ppc64le ppc64le ppc64le GNU/Linux

Guest (Fedora 26) Kernel:

uname -a

Linux localhost.localdomain 4.12.13-300.fc26.ppc64le #1 SMP Thu Sep 14 15:47:43 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux



<cde:info> Mirrored with LTC bug #159384 </cde:info>
balamuruhans commented 7 years ago

Attachment: libvirtd debug logs of source and destination: libvirtd_source.tar.gz libvirtd_destination.tar.gz

bssrikanth commented 7 years ago

we will try migration with qemu 2.10 will get back here..

bssrikanth commented 7 years ago

We had got in wrong qemu version. With QEMU 2.10 we are not hitting any issues with migration. Closing this bug.