Open ader1990 opened 1 day ago
Try adding console=tty0
to grub.cfg.
To give more context what happens, as there are two scenarios that behave a little bit different.
This scenario can be easily reproduced on an ARM64 box (DO NOT USE emulation), with the command:
bash flatcar_production_qemu_uefi.sh -nographic -vnc :1 -device virtio-gpu-pci -device virtio-keyboard -device virtio-mouse
In this scenario, after the VM is started and a VNC console is opened, one can see in the VNC console, in this order: the TianoCore bootloader, Grub 2 with the Flatcar boot options (default, usr-a, usr-b), Booting Flatcar default
message, and if everything went well, after some time, the login console.
NOTE that between the Booting Flatcar default
message and the login console, the VNC console will display a message Display output is not active
or Guest has not initialized the display output yet
and sometimes, for a short time, a blinking _ (underscore) on the top left part of the screen.
Example Libvirt XML:
<domain type='kvm' id='5'>
<name>flatcararm64</name>
<uuid>58d48567-0cf0-448c-a519-03ccb030efd0</uuid>
<metadata>
<libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
<libosinfo:os id="http://libosinfo.org/linux/2022"/>
</libosinfo:libosinfo>
</metadata>
<memory unit='KiB'>2097152</memory>
<currentMemory unit='KiB'>2097152</currentMemory>
<vcpu placement='static'>4</vcpu>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='aarch64' machine='virt-6.2'>hvm</type>
<loader readonly='yes' type='pflash'>/var/lib/libvirt/images/flatcar-bootloader.img</loader>
<nvram>/var/lib/libvirt/images/flatcar-nvram.img</nvram>
<boot dev='hd'/>
<bootmenu enable='yes'/>
</os>
<features>
<acpi/>
<gic version='3'/>
</features>
<cpu mode='host-passthrough' check='none'/>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<emulator>/usr/bin/qemu-system-aarch64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/var/lib/libvirt/images/flatcar.qcow2' index='1'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</disk>
<controller type='usb' index='0' model='qemu-xhci' ports='15'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='0' model='pcie-root'>
<alias name='pcie.0'/>
</controller>
<controller type='pci' index='1' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='1' port='0x8'/>
<alias name='pci.1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='2' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='2' port='0x9'/>
<alias name='pci.2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='3' port='0xa'/>
<alias name='pci.3'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<controller type='pci' index='4' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='4' port='0xb'/>
<alias name='pci.4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
</controller>
<controller type='pci' index='5' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='5' port='0xc'/>
<alias name='pci.5'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
</controller>
<controller type='pci' index='6' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='6' port='0xd'/>
<alias name='pci.6'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
</controller>
<controller type='pci' index='7' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='7' port='0xe'/>
<alias name='pci.7'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
</controller>
<controller type='pci' index='8' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='8' port='0xf'/>
<alias name='pci.8'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
</controller>
<controller type='pci' index='9' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='9' port='0x10'/>
<alias name='pci.9'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
</controller>
<controller type='pci' index='10' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='10' port='0x11'/>
<alias name='pci.10'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
</controller>
<controller type='pci' index='11' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='11' port='0x12'/>
<alias name='pci.11'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
</controller>
<controller type='pci' index='12' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='12' port='0x13'/>
<alias name='pci.12'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
</controller>
<controller type='pci' index='13' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='13' port='0x14'/>
<alias name='pci.13'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
</controller>
<controller type='pci' index='14' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='14' port='0x15'/>
<alias name='pci.14'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
</controller>
<controller type='pci' index='15' model='pcie-root-port'>
<model name='pcie-root-port'/>
<target chassis='15' port='0x16'/>
<alias name='pci.15'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
</controller>
<controller type='pci' index='16' model='pcie-to-pci-bridge'>
<model name='pcie-pci-bridge'/>
<alias name='pci.16'/>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='virtio-serial' index='0'>
<alias name='virtio-serial0'/>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<serial type='pty'>
<source path='/dev/pts/3'/>
<target type='system-serial' port='0'>
<model name='pl011'/>
</target>
<alias name='serial0'/>
</serial>
<console type='pty' tty='/dev/pts/3'>
<source path='/dev/pts/3'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<channel type='unix'>
<source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-5-flatcararm64/org.qemu.guest_agent.0'/>
<target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='tablet' bus='usb'>
<alias name='input0'/>
<address type='usb' bus='0' port='1'/>
</input>
<input type='keyboard' bus='usb'>
<alias name='input1'/>
<address type='usb' bus='0' port='2'/>
</input>
<tpm model='tpm-tis'>
<backend type='emulator' version='2.0'/>
<alias name='tpm0'/>
</tpm>
<graphics type='vnc' port='5901' autoport='no' listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
</graphics>
<audio id='1' type='none'/>
<video>
<model type='virtio' vram='16384' heads='1' primary='yes'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x10' slot='0x01' function='0x0'/>
</video>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
</memballoon>
<rng model='virtio'>
<backend model='random'>/dev/urandom</backend>
<alias name='rng0'/>
<address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
</rng>
</devices>
<seclabel type='dynamic' model='apparmor' relabel='yes'>
<label>libvirt-58d48567-0cf0-448c-a519-03ccb030efd0</label>
<imagelabel>libvirt-58d48567-0cf0-448c-a519-03ccb030efd0</imagelabel>
</seclabel>
<seclabel type='dynamic' model='dac' relabel='yes'>
<label>+64055:+108</label>
<imagelabel>+64055:+108</imagelabel>
</seclabel>
</domain>
Curiously, the video device as set in the libvirt config xml translates to a qemu-kvm command -device virtio-vga,id=video0,max_outputs=1,bus=pci.16,addr=0x1
.
NOTE that in scenario 2, the message is Guest has not initialized the display output yet
and sometimes, for a short time, a blinking _ (underscore) on the top left part of the screen. Sometimes, there is no bootloader (TianoCore and grub display).
More debug is required.
Try adding
console=tty0
to grub.cfg.
The ARM64 ttyAMA0 is already set correctly in the bootengine. This most likely looks like an initrd issue with either a missing kernel module not built or kernel parameter or even systemd parameter.
Try adding
console=tty0
to grub.cfg.The ARM64 ttyAMA0 is already set correctly in the bootengine. This most likely looks like an initrd issue with either a missing kernel module not built or kernel parameter or even systemd parameter.
My bad, it is set correctly inhere: https://github.com/flatcar/scripts/blob/main/build_library/grub.cfg#L82
ttyAM0 is a serial console, vnc connects to a graphical console. If you want to see boot output on both you need to set tty0 as well
ttyAM0 is a serial console, vnc connects to a graphical console. If you want to see boot output on both you need to set tty0 as well
Unfortunately, it does not work, at least on my environment - I tried to add the console=tty0
in the /oem/grub.cfg, or with the ignition.ign at first boot or in the grub bootloader screen edit, the same result, during the initrd stage, there was no output to the VNC console. There is output during the bootloaders, and after the initrd pivot, but not on the initrd stage. Maybe there is another place to add the setting?
Description
On ARM64, when booting on QEMU-KVM with VNC enabled, the VNC console does not show any output in the Flatcar initrd stage early boot stage.
Impact
No way to debug from VNC if any ignition or initrd stage error appears.
Environment and steps to reproduce
TBD.
Additional information
Please add any information here that does not fit the above format.