mister-Q
commented
1 year ago oups sorry for the fomatting:
some parts of libvirt xml is missing:
Open mister-Q opened 1 year ago
mister-Q
commented
1 year ago oups sorry for the fomatting:
some parts of libvirt xml is missing:
mister-Q
commented
1 year ago `
`
Wenzel
commented
1 year ago Hi, thank you for posting your issue. Can you reformat your post with Markdown ? It's very hard to read right now.
Then I can take a look.
mister-Q
commented
1 year ago ok no pb. Hello all, I am trying to setup a introspection env by following the setup guide: https://kvm-vmi.github.io/kvm-vmi/master/setup.html and some other blogs.
I managed to compile kvm/qemu/libkvmi but now i am stuck with the following issues:
Any help will be appreciated :)
My setup is :
The host
kvm installation
$ git checkout kvmi-v12
$ scripts/config --enable KVM
$ scripts/config --enable KVM_INTEL
$ scripts/config --enable KVM_INTROSPECTION
$ scripts/config --enable REMOTE_MAPPING **
$ scripts/config --disable TRANSPARENT_HUGEPAGE **
$ scripts/config --disable SYSTEM_TRUSTED_KEYS
$ scripts/config --disable MODULE_SIG_KEY
$ scripts/config --disable SECURITY_APPARMOR **
(/!\ I a m not confident, or misunderstood the reasons of the options suffixed with **
-> verify grub config and boot is OK
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.0-rc2+ root=UUID=c8b585d2-0585-4e0b-8b9f-af4ad002cde0 ro quiet kvm.introspection=1
qemu installation:
$ git checkout kvmi-v12
$ ./configure --target-list=x86_64-softmmu --enable-spice --prefix=/usr
--> this will replace the version from the debian package
libkvmi
--> compilation/installation seems OK
The guest
libvirt
$ virt-install --virt-type kvm --name buster-amd64 --cdrom ~/vms/debian-11.5.0-amd64-netinst.iso --os-variant debian11 --disk size=10 --memory 1000
--> no other modification after net installation and reboot
xml modifications and extract
$ virsh edit buster-amd64
<qemu:commandline
<qemu:arg value='-chardev'/>
<qemu:arg value='socket,path=/tmp/introspector,id=chardev0,reconnect=10'/>
<qemu:arg value='-object'/>
<qemu:arg value='introspection,id=kvmi,chardev=chardev0'/>
</qemu:commandline>
....
<vcpu placement='static'>2</vcpu>
<os>
<type arch='x86_64' machine='pc-q35-4.2'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<vmport state='off'/>
</features>
<cpu mode='host-model' check='partial'/>The first test and the error
libkvmi
$ /hookguest-libkvmi /tmp/introspector
Waiting...
New handshake: name 'buster-amd64' start_time 1670406989
New guest: 05 4c 35 55 50 94 46 5f 9e 16 16 47 08 e6 0e ee fd 6 ctx (nil)
kvmi_get_vcpu_count: Invalid argument--> so first handshake works, but then fails
The second test and the error
When i try to use the selftest of kvm (got this from the vagrantfile playbook )
kvm-vmi/kvm/tools/testing/selftests/kvm/x86_64$ ./kvmi_test
--> no output
/var/log/auth.log
Dec 7 11:34:58 bullseye kernel: [ 7534.533799] ------------[ cut here ]------------
Dec 7 11:34:58 bullseye kernel: [ 7534.533804] Invalid MSR 140, please adapt vmx_possible_passthrough_msrs[]
Dec 7 11:34:58 bullseye kernel: [ 7534.533808] WARNING: CPU: 7 PID: 47451 at arch/x86/kvm/vmx/vmx.c:585 is_valid_passthrough_msr+0x5d/0xe0
Dec 7 11:34:58 bullseye kernel: [ 7534.533817] Modules linked in: vsock_loopback(E) vmw_vsock_virtio_transport_common(E) vmw_vsock_vmci_transport(E) vsock(E) vmw_vmci(E) intel_rapl_msr(E) intel_rapl_common(E) snd_hda_codec_hdmi(E) snd_hda_codec_realtek(E) x86_pkg_temp_thermal(E) snd_hda_codec_generic(E) intel_powerclamp(E) ledtrig_audio(E) hid_generic(E) coretemp(E) usbhid(E) hid(E) snd_hda_intel(E) ghash_clmulni_intel(E) mei_wdt(E) mei_hdcp(E) snd_intel_dspcfg(E) snd_intel_sdw_acpi(E) snd_hda_codec(E) aesni_intel(E) libaes(E) snd_hda_core(E) crypto_simd(E) snd_pcsp(E) cryptd(E) snd_hwdep(E) snd_pcm(E) rapl(E) snd_timer(E) intel_cstate(E) iTCO_wdt(E) intel_pmc_bxt(E) mei_me(E) iTCO_vendor_support(E) snd(E) at24(E) sg(E) watchdog(E) mei(E) intel_uncore(E) soundcore(E) serio_raw(E) evdev(E) fuse(E) sunrpc(E) configfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc32c_generic(E) crc16(E) mbcache(E) jbd2(E) sd_mod(E) t10_pi(E) crc_t10dif(E) sr_mod(E) crct10dif_generic(E) cdrom(E) ahci(E) libahci(E) libata(E) i915(E)
Dec 7 11:34:58 bullseye kernel: [ 7534.533868] scsi_mod(E) i2c_algo_bit(E) scsi_common(E) crct10dif_pclmul(E) ttm(E) crct10dif_common(E) xhci_pci(E) drm_kms_helper(E) crc32_pclmul(E) e1000e(E) xhci_hcd(E) cec(E) ehci_pci(E) ptp(E) i2c_i801(E) pps_core(E) psmouse(E) ehci_hcd(E) crc32c_intel(E) i2c_smbus(E) drm(E) lpc_ich(E) usbcore(E) video(E) button(E)
Dec 7 11:34:58 bullseye kernel: [ 7534.533887] CPU: 7 PID: 47451 Comm: kvmi_test Tainted: G W E 5.15.0-rc2+ https://github.com/KVM-VMI/kvm-vmi/issues/1
Dec 7 11:34:58 bullseye kernel: [ 7534.533890] Hardware name: Dell Inc. OptiPlex 7010/0KRC95, BIOS A12 01/10/2013
Dec 7 11:34:58 bullseye kernel: [ 7534.533891] RIP: 0010:is_valid_passthrough_msr+0x5d/0xe0
Dec 7 11:34:58 bullseye kernel: [ 7534.533896] Code: c7 c1 94 0e 81 93 ba 48 00 00 00 eb 06 8b 10 48 83 c0 04 39 d7 74 23 48 39 c1 75 f1 89 fe 48 c7 c7 70 34 af 93 e8 36 6f 87 00 <0f> 0b 31 c0 c3 8d 87 40 f9 ff ff 83 f8 1f 77 be b8 01 00 00 00 c3
Dec 7 11:34:58 bullseye kernel: [ 7534.533898] RSP: 0018:ffffa9d1c221bd00 EFLAGS: 00010286
Dec 7 11:34:58 bullseye kernel: [ 7534.533900] RAX: 0000000000000000 RBX: 0000000000000140 RCX: ffff939f923d8a08
Dec 7 11:34:58 bullseye kernel: [ 7534.533902] RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffff939f923d8a00
Dec 7 11:34:58 bullseye kernel: [ 7534.533904] RBP: 0000000000000002 R08: 0000000000000000 R09: ffffa9d1c221bb28
Dec 7 11:34:58 bullseye kernel: [ 7534.533905] R10: ffffa9d1c221bb20 R11: ffffffff940dede8 R12: 0000000000000002
Dec 7 11:34:58 bullseye kernel: [ 7534.533906] R13: ffff939ee013f000 R14: ffff939f0e624000 R15: dead000000000100
Dec 7 11:34:58 bullseye kernel: [ 7534.533908] FS: 00007fa7d1373700(0000) GS:ffff939f923c0000(0000) knlGS:0000000000000000
Dec 7 11:34:58 bullseye kernel: [ 7534.533910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Dec 7 11:34:58 bullseye kernel: [ 7534.533912] CR2: 0000000000000000 CR3: 00000001214b2005 CR4: 00000000001726e0
Dec 7 11:34:58 bullseye kernel: [ 7534.533913] Call Trace:
Dec 7 11:34:58 bullseye kernel: [ 7534.533917] vmx_enable_intercept_for_msr+0x3a/0x160
Dec 7 11:34:58 bullseye kernel: [ 7534.533922] kvmi_control_msrw_intercept+0x28/0x40
Dec 7 11:34:58 bullseye kernel: [ 7534.533926] handle_vcpu_control_msr+0x84/0x90
Dec 7 11:34:58 bullseye kernel: [ 7534.533929] kvmi_job_vcpu_msg+0x55/0x80
Dec 7 11:34:58 bullseye kernel: [ 7534.533931] kvmi_run_jobs+0x86/0xd0
Dec 7 11:34:58 bullseye kernel: [ 7534.533934] kvmi_handle_requests+0x2e/0xc0
Dec 7 11:34:58 bullseye kernel: [ 7534.533936] kvm_arch_vcpu_ioctl_run+0x7c1/0x1560
Dec 7 11:34:58 bullseye kernel: [ 7534.533939] kvm_vcpu_ioctl+0x274/0x660
Dec 7 11:34:58 bullseye kernel: [ 7534.533943] __x64_sys_ioctl+0x83/0xb0
Dec 7 11:34:58 bullseye kernel: [ 7534.533949] do_syscall_64+0x3b/0xc0
Dec 7 11:34:58 bullseye kernel: [ 7534.533954] entry_SYSCALL_64_after_hwframe+0x44/0xae
Dec 7 11:34:58 bullseye kernel: [ 7534.533958] RIP: 0033:0x7fa7d16705f7
Dec 7 11:34:58 bullseye kernel: [ 7534.533961] Code: 00 00 00 48 8b 05 99 c8 0d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 69 c8 0d 00 f7 d8 64 89 01 48
Dec 7 11:34:58 bullseye kernel: [ 7534.533963] RSP: 002b:00007fa7d1372e58 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Dec 7 11:34:58 bullseye kernel: [ 7534.533965] RAX: ffffffffffffffda RBX: 0000000000e7d790 RCX: 00007fa7d16705f7
Dec 7 11:34:58 bullseye kernel: [ 7534.533967] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000007
Dec 7 11:34:58 bullseye kernel: [ 7534.533968] RBP: 0000000000000001 R08: 000000000041339e R09: 0000000000000001
Dec 7 11:34:58 bullseye kernel: [ 7534.533969] R10: 00007fa7d13739d0 R11: 0000000000000246 R12: 00007ffd3296658e
Dec 7 11:34:58 bullseye kernel: [ 7534.533970] R13: 0000000000e7b2a0 R14: 00007fa7d1372fc0 R15: 0000000000e7d820
Dec 7 11:34:58 bullseye kernel: [ 7534.533973] ---[ end trace 955aa6e686a1980f ]---Thanks
You checkout kvmi-12 which is a version created for upstream integration, but not tested very well.
Please follow these instructions to get the stable branches:
mister-Q
commented
1 year ago Thx.
indeed it works better... but sitll have an issue with
$ kvmi_test
KVM_CAP_INTROSPECTION not available, skipping testbut libkvmi example looks OK
$ ./hookguest-libkvmi /tmp/introspector
Waiting...
New handshake: name 'buster-amd64' start_time 481
New guest: 05 4c 35 55 50 94 46 5f 9e 16 16 47 08 e6 0e ee fd 6 ctx (nil)
Sending the pause command...
We should receive 2 pause events
Max gfn: 0x3e800
Waiting...
Pop event
PAUSE (vcpu0)
///skip i will now try to get further with libvmi, hope this capability missing is not an issue.
Thx,
Hello all, I am trying to setup a introspection env by following the setup guide: https://kvm-vmi.github.io/kvm-vmi/master/setup.html and some other blogs.
I managed to compile kvm/qemu/libkvmi but now i am stuck with the following issues:
Any help will be appriacted :)
My setup is:
Host:
for kvm: git checkout kvmi-v12 then custom kernel compilation with specific options: (from blog/vagrant and setup) scripts/config --enable KVM scripts/config --enable KVM_INTEL scripts/config --enable KVM_INTROSPECTION scripts/config --enable REMOTE_MAPPING scripts/config --disable TRANSPARENT_HUGEPAGE scripts/config --disable SYSTEM_TRUSTED_KEYS scripts/config --disable MODULE_SIG_KEY scripts/config --disable SECURITY_APPARMOR (/!\ I a m not confident, or misunderstood the reasons of the options sufixed with
$ cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-5.15.0-rc2+ root=UUID=c8b585d2-0585-4e0b-8b9f-af4ad002cde0 ro quiet kvm.introspection=1
for qemu: $ git checkout kvmi-v12 $ ./configure --target-list=x86_64-softmmu --enable-spice --prefix=/usr this will replace the version from the debian package ** qemu/libvirt
extract from $ virsh edit buster-amd64
....