Packer 1.8.x regression: qemu builds fail due to missing output-qemu/packer-qemu

haidangwa commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

When filing a bug, please include the following headings if possible. Any example text in this template can be deleted.

Overview of the Issue

This appears to be a regression, as I'm seeing close similarities to hashicorp/packer#10135, but I'm seeing the problem with packer 1.8.3 on Ubuntu 20.04. I downgraded to packer 1.6.5 and it was able to complete the build.

Reproduction Steps

git clone git@github.com:canonical/packer-maas.git
cd packer-maas/ubuntu
sudo make all

Packer version

From packer version 1.8.3

Simplified Packer Template

https://github.com/canonical/packer-maas/blob/master/ubuntu/ubuntu-cloudimg.pkr.hcl

Operating system and Environment details

ubuntu@valid-yak:~/packer-maas/ubuntu$ uname -a
Linux valid-yak 5.4.0-122-generic hashicorp/packer#138-Ubuntu SMP Wed Jun 22 15:00:31 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Log Fragments and crash.log files

==> qemu.lvm: Starting VM, booting from CD-ROM
    qemu.lvm: view the screen of the VM, connect via VNC without a password to
    qemu.lvm: vnc://127.0.0.1:5901
    qemu.lvm: The VM will be run headless, without a GUI. If you want to
2022/08/09 18:21:58 packer-plugin-qemu_v1.0.5_x5.0_linux_amd64 plugin: 2022/08/09 18:21:58 Qemu Builder has no floppy files, not attaching a floppy.
    qemu.lvm: view the screen of the VM, connect via VNC without a password to
    qemu.lvm: vnc://127.0.0.1:5901
==> qemu.lvm: Overriding default Qemu arguments with qemuargs template option...
2022/08/09 18:21:58 packer-plugin-qemu_v1.0.5_x5.0_linux_amd64 plugin: 2022/08/09 18:21:58 Executing /usr/bin/qemu-system-x86_64: []string{"-boot", "once=d", "-machine", "type=pc,accel=kvm", "-smp", "cpus=2,sockets=2", "-name", "packer-lvm", "-vnc", "127.0.0.1:1", "-vga", "qxl", "-device", "virtio-blk-pci,drive=drive0,bootindex=0", "-device", "virtio-blk-pci,drive=cdrom0,bootindex=1", "-device", "virtio-blk-pci,drive=drive1,bootindex=2", "-device", "virtio-net,netdev=user.0", "-drive", "if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd", "-drive", "if=pflash,format=raw,file=OVMF_VARS.fd", "-drive", "file=output-qemu/packer-qemu,if=none,id=drive0,cache=writeback,discard=ignore,format=raw", "-drive", "file=seeds-lvm.iso,format=raw,cache=none,if=none,id=drive1", "-drive", "file=packer_cache/ubuntu.iso,if=none,id=cdrom0,media=cdrom", "-netdev", "user,id=user.0,hostfwd=tcp::4018-:22", "-m", "1024M"}
2022/08/09 18:21:58 packer-plugin-qemu_v1.0.5_x5.0_linux_amd64 plugin: 2022/08/09 18:21:58 Started Qemu. Pid: 27401
2022/08/09 18:21:58 packer-plugin-qemu_v1.0.5_x5.0_linux_amd64 plugin: 2022/08/09 18:21:58 Qemu stderr: qemu-system-x86_64: -drive file=output-qemu/packer-qemu,if=none,id=drive0,cache=writeback,discard=ignore,format=raw: Could not open 'output-qemu/packer-qemu': No such file or directory
2022/08/09 18:21:58 packer-plugin-qemu_v1.0.5_x5.0_linux_amd64 plugin: 2022/08/09 18:21:58 failed to unlock port lockfile: close tcp 127.0.0.1:5901: use of closed network connection
==> qemu.lvm: Error launching VM: Qemu failed to start. Please run with PACKER_LOG=1 to get more info.
2022/08/09 18:21:58 packer-plugin-qemu_v1.0.5_x5.0_linux_amd64 plugin: 2022/08/09 18:21:58 failed to unlock port lockfile: close tcp 127.0.0.1:4018: use of closed network connection

nywilken commented 2 years ago

Hi there @haidangwa thanks for reporting this issue. Could you provide a full debug log so that we can see what steps are not happening before the failure, please.

That said does using an older version of the QEMU plugin work?

If you could try downgrading the plugin to v1.0.0 by running packer plugins install github.com/hashicorp/qemu v1.0.0 and run your build again that would be helpful in determining where the issues lies. There is a bit of difference between Packer 1.6.5 and 1.8.x as it pertains to Packer core. But not so many differences to the QEMU plugin.

I'm going to move this issue to the QEMU Packer plugin repo so we can track it there.

haidangwa commented 2 years ago

with packer 1.8.3 and packer qemu plugin 1.0.0, here's the output:

==> qemu.lvm: Looking for available port between 5900 and 6000 on 127.0.0.1
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Found available port: 5911 on IP: 127.0.0.1
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Found available VNC port: 5911 on IP: 127.0.0.1
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Qemu --version output: QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.23)
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Qemu version: 4.2.1
==> qemu.lvm: Starting VM, booting from CD-ROM
    qemu.lvm: view the screen of the VM, connect via VNC without a password to
    qemu.lvm: vnc://127.0.0.1:5911
    qemu.lvm: The VM will be run headless, without a GUI. If you want to
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Qemu Builder has no floppy files, not attaching a floppy.
    qemu.lvm: view the screen of the VM, connect via VNC without a password to
    qemu.lvm: vnc://127.0.0.1:5911
==> qemu.lvm: Overriding default Qemu arguments with qemuargs template option...
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Executing /usr/bin/qemu-system-x86_64: []string{"-vnc", "127.0.0.1:11", "-name", "packer-lvm", "-netdev", "user,id=user.0,hostfwd=tcp::2231-:22", "-m", "1024M", "-boot", "once=d", "-machine", "type=pc,accel=kvm", "-vga", "qxl", "-device", "virtio-blk-pci,drive=drive0,bootindex=0", "-device", "virtio-blk-pci,drive=cdrom0,bootindex=1", "-device", "virtio-blk-pci,drive=drive1,bootindex=2", "-device", "virtio-net,netdev=user.0", "-drive", "if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE.fd", "-drive", "if=pflash,format=raw,file=OVMF_VARS.fd", "-drive", "file=output-qemu/packer-qemu,if=none,id=drive0,cache=writeback,discard=ignore,format=raw", "-drive", "file=seeds-lvm.iso,format=raw,cache=none,if=none,id=drive1", "-drive", "file=packer_cache/ubuntu.iso,if=none,id=cdrom0,media=cdrom", "-smp", "cpus=2,sockets=2"}
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Started Qemu. Pid: 9778
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 Qemu stderr: qemu-system-x86_64: -drive file=output-qemu/packer-qemu,if=none,id=drive0,cache=writeback,discard=ignore,format=raw: Could not open 'output-qemu/packer-qemu': No such file or directory
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 failed to unlock port lockfile: close tcp 127.0.0.1:5911: use of closed network connection
==> qemu.lvm: Error launching VM: Qemu failed to start. Please run with PACKER_LOG=1 to get more info.
2022/08/09 22:15:24 packer-plugin-qemu_v1.0.0_x5.0_linux_amd64 plugin: 2022/08/09 22:15:24 failed to unlock port lockfile: close tcp 127.0.0.1:2231: use of closed network connection
==> qemu.lvm: Deleting output directory...
2022/08/09 22:15:24 [INFO] (telemetry) ending qemu.lvm
==> Wait completed after 3 seconds 270 milliseconds
2022/08/09 22:15:24 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2022/08/09 22:15:24 machine readable: qemu.lvm,error []string{"Build was halted."}
Build 'qemu.lvm' errored after 3 seconds 270 milliseconds: Build was halted.
==> Builds finished but no artifacts were created.

2022/08/09 22:15:24 [INFO] (telemetry) Finalizing.
==> Wait completed after 3 seconds 270 milliseconds

==> Some builds didn't complete successfully and had errors:
--> qemu.lvm: Build was halted.

haidangwa commented 2 years ago

here's the gist of debug log: packer 1.8.3 and packer-qemu-plugin 1.0.0

https://gist.github.com/haidangwa/cb65a742b784a3ada19d422875104430

haidangwa commented 2 years ago

The failure occurs from packer 1.7.0 - 1.8.3. It was only successful with packer 1.6.6 and 1.6.5.

hashicorp / packer-plugin-qemu