Open mcastelino opened 6 years ago
Excellent info - it's going to be great to get this (and other info) translated into a 'how to debug' reference .md file. Note for whoever ends up doing the translation - please pull the information out of the gist @mcastelino references into the document.
How to debug QEMU or Kernel issues for Clear Containers
Manual command line for Clear Container Q35 Launch. This lets you debug w/o needing the entire kubernetes or docker framework.
If you have clear containers installed you can get the kernel and rootfs image from /usr/share/clearcontainers
For q35 machine time
For PC Machine Type
Note the use of the compressed kernel in this case
Tracing QEMU/KVM interactions
If the console log does not provide enough information, then it is time to trace the KVM and QEMU interactions.
Details on how to do that can be found here
https://gist.github.com/mcastelino/b31f0648707b25478eb2a44f94a861fd
QEMU Console debug
If that also does not suffice, then you need to introspect the VM using the QEMU monitor.
The easiest way to do the same would be over telnet
Append the following to the QEMU command line
Now you can telnet to access the monitor
The qemu monitor provides a variety of info commands to provide machine context. The full list can be obtained using the info command
The following are normally most useful
GDB
If you are still in a pickle... then its time to gdb the guest kernel. Ideally you want access to the QEMU console at the same time
Starting QEMU with gdb enabled
Append
to the QEMU command line. This will start QEMU with gdb enabled, but will not stop the execution of the kernel.
You can now attach gdb to the running kernel (which will stop execution at the time you attach
Note: vmlinux is the uncompressed kernel Note: If you start with the option "-s -S", then the kernel will be started in the stopped state.
Mapping instructions to the code
The Clear Containers kernel does not have CONFIG_DEBUG_INFO=y in the .config. For quick and dirty mapping you can perform
To get the code listing. This is helpful to figure out where issues may be.