search

microsoft / openvmm

Home of OpenVMM and OpenHCL.
http://openvmm.dev/
MIT License
1.55k stars 79 forks source link

tdx: cargo flag `--with-perf-tools does` not include perf tooling in the ramdisk #357

Open babayet2 opened 3 hours ago

babayet2 commented 3 hours ago

Scenario Building CVMs with perf tooling for evaluating OHCL TDX performance

Expected Behavior Running the below command should produce an initrd which contains the "perf" and "tracing" tools, which we can leverage to get kernel (and user?) stacks. cargo xflowey build-igvm x64-cvm-devkern --release --with-perf-tools --override-manifest=./vm/loader/manifests/openhcl-x64-cvm-dev.json

Observed Behavior We must modify openhcl's rootfs.config to include the perf and tracing binaries, a build a custom kernel with CONFIG_BPF_SYSCALL. With this workaround we are able to get flamegraphs of the kernel stacks, but we have not found a build configuration that allows us to get the more critical user stacks.

chris-oo commented 2 hours ago

@jaredwhitedev maybe knows some hints here?

jaredwhitedev commented 2 hours ago

Yeah the --with-perf-tools flag didn't work last I checked. If you dump a working perf binary into the running OHCL environment and have debug info enabled (--with-debuginfo), perf record will work as expected though. No kernel config changes needed.

chris-oo commented 34 minutes ago

I wonder if we need to remove that flag or just have that flag bundle some stock perf tools image from a known source. Where did you get your "working perf binary"?

I assume we still need to add CONFIG_BPF_SYSCALL for kernel flamegraph support?

jaredwhitedev commented 15 minutes ago

I don't remember at this point, I think I built it a long time ago. Important bit is that it is statically linked. I believe we already bundle the perf binary with the kernel build, I see it here: flowey-persist/flowey_lib_hvlite__download_openhcl_kernel_package/extracted/Microsoft.OHCL.Kernel.6.6.51.7-main-cvm-x64.tar.gz/tools/perf/bin/perf. The --with-perf-tools flag is probably just broken.

I haven't needed CONFIG_BPF_SYSCALL for kernel and user stack sampling with perf.