ltp: test case - Githubissues

Martins3 commented 3 years ago

[x] 还有，按道理来说，即使是将 dune_enter 放到 parent process 上，也不应该出现 pr_err("KVM_CREATE_VM"); 的错误。

Martins3 commented 3 years ago

[x] timer_create03 cause some stable problems
[x] sudo ./dune-sethostname01

Martins3 commented 3 years ago

[x] /home/maritns3/core/4000/ltp/testcases/kernel/syscalls/nftw/nftw64.c need to add dune_enter();

Martins3 commented 3 years ago

[ ] runtest/syscalls-ipc 's relation with runtest/syscalls ?

Martins3 commented 3 years ago

[x] remove the stupid file debug

Martins3 commented 3 years ago

Normal hypercall:

[616473.462028] kvm [25170]: VCPU Register Dump:
[616473.462030] kvm [25170]:    pc = 0x980000fff77541d0
[616473.462031] kvm [25170]:    exceptions: 00000000
[616473.462034] kvm [25170]:    gpr00: 00000000 00000001 00001455 fff7765f20
[616473.462036] kvm [25170]:    gpr04: 00000000 00000000 fff775ef30 00000000
[616473.462039] kvm [25170]:    gpr08: fff76db820 fff7714030 fff7762f20 00000003
[616473.462041] kvm [25170]:    gpr12: fffbc2dbe8 00000008 00000008 fff7761e28
[616473.462043] kvm [25170]:    gpr16: fff76ccca8 fff76ccca8 00000000 00000002
[616473.462047] kvm [25170]:    gpr20: fff76d2580 00000001 fff76d6580 00000000
[616473.462049] kvm [25170]:    gpr24: 00000000 fff75d97a0 12bc8c058 00000000
[616473.462051] kvm [25170]:    gpr28: fff76db820 fffbc2dd60 00000000 fff7552c4c
[616473.462053] kvm [25170]:    hi: 0x00000002
[616473.462055] kvm [25170]:    lo: 0x6666666666666667
[616473.462057] kvm [25170]:    Status: 0x740000a2, Cause: 0x40008020
[616473.462059] kvm [25170]:    EPC: 0xfff75d97cc
[616473.462061] kvm [25170]: guest badvaddr 1200084cc
[616473.462063] kvm [25170]: guest entryhi120008000
[616473.462065] kvm [25170]: huxueshi : what's host_cp0_badinstr
[616473.462067] kvm [25170]: update_pc(): New PC: 0x980000fff77541d4
[616473.462069] kvm [25170]: [0x980000fff77541d4] HYPCALL 0x0
[616473.462071] kvm [25170]: huxueshi trace leave kvm_vz_vcpu_run:3565

Martins3 commented 3 years ago

FAIL

[616634.351455] kvm [25556]: VCPU Register Dump:
[616634.351457] kvm [25556]:    pc = 0x980000fff43501ac
[616634.351459] kvm [25556]:    exceptions: 00000000
[616634.351462] kvm [25556]:    gpr00: 00000000 00000001 00001455 fff45e2740
[616634.351464] kvm [25556]:    gpr04: 00000000 00000000 fff45db750 00000001
[616634.351467] kvm [25556]:    gpr08: fff453b820 fff4590030 fff45def20 00000004
[616634.351470] kvm [25556]:    gpr12: 00000006 00000001 00000008 fff45dde28
[616634.351472] kvm [25556]:    gpr16: fff452cca8 fff452cca8 00000000 00000002
[616634.351475] kvm [25556]:    gpr20: fff4532580 00000001 fff4536580 00000000
[616634.351477] kvm [25556]:    gpr24: 00000000 fff44397a0 00000ff3 00000003
[616634.351480] kvm [25556]:    gpr28: fff453b820 fffba1ed10 00000000 fff43b2c4c
[616634.351481] kvm [25556]:    hi: 0x00000002
[616634.351484] kvm [25556]:    lo: 0x6666666666666667
[616634.351486] kvm [25556]:    Status: 0x740000a2, Cause: 0x4000800c
[616634.351488] kvm [25556]:    EPC: 0xfff44397cc
[616634.351490] kvm [25556]: guest badvaddr 12760c058
[616634.351491] kvm [25556]: guest entryhi12760c000
[616634.351493] kvm [25556]: huxueshi : what's host_cp0_badinstr
[616634.351495] kvm [25556]: update_pc(): New PC: 0x980000fff43501b0
[616634.351498] kvm [25556]: [0x980000fff43501b0] HYPCALL 0x16
[616634.351500] kvm [25556]: huxueshi trace leave kvm_vz_vcpu_run:3565
➜  timer_create git:(master)

Martins3 commented 3 years ago

➜  test git:(main) ✗ rm -f fork && make && ./fork
make -C ../dune
make[1]: Entering directory '/home/loongson/dune/dune'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/loongson/dune/dune'
/usr/local/musl/bin/musl-gcc -static fork.c ../dune/libdune.a -o fork
  Info: open /dev/kvm
  Info: KVM_GET_API_VERSION
  Info: KVM_CREATE_VM
  Info: KVM_SET_USER_MEMORY_REGION
  Info: ebase address : fff69b0000
--- Process Map Dump ---
heee
after open
after fread
after setvbuf
0x0000000120020000-0xffffffffffffffff ---S ffffffffffffffff
0x0000000120020000-0x0000000120024000 RWXP 00020000 /home/loongson/dune/test/fork
0x000000012bc5c000-0x000000012bc60000 ---P 00000000 [heap]
0x000000012bc60000-0x000000012bc64000 RWXP 00000000 [heap]
0x000000fff69ac000-0x000000fff69b4000 RWXP 00000000
0x000000fff69b4000-0x000000fff69bc000 RWXS 00000000 anon_inode:kvm-vcpu:0
0x000000fff69bc000-0x000000fff69c0000 RWXP 00000000
0x000000fffbc1c000-0x000000fffbc40000 RWXP 00000000 [stack]
0x000000fffbff4000-0x000000fffbff8000 R-XP 00000000
0x000000fffeaf4000-0x000000fffeafc000 R--P 00000000 [vvar]
0x000000fffeafc000-0x000000fffeb00000 R-XP 00000000 [vdso]
 Error: vcpu_id=0
KVM_EXIT_IS_NOT_HYPERCALL: No error information
  Info: open /dev/kvm
  Info: KVM_GET_API_VERSION
  Info: KVM_CREATE_VM
  Info: KVM_SET_USER_MEMORY_REGION
 Error: vcpu_id=1
KVM_EXIT_IS_NOT_HYPERCALL: No error information

[622018.486115] kvm [13748]: VCPU Register Dump:
[622018.486117] kvm [13748]:    pc = 0x980000fff69b01ac
[622018.486119] kvm [13748]:    exceptions: 00000000
[622018.486122] kvm [13748]:    gpr00: 00000000 20000000 0000143a 1200214c0
[622018.486125] kvm [13748]:    gpr04: 0000143a 12001e648 120028038 00000000
[622018.486127] kvm [13748]:    gpr08: 1200110a0 00000006 fffffffffffffff0 00000006
[622018.486129] kvm [13748]:    gpr12: 120020000 fffeb00000 ffffffffffffffff 00000030
[622018.486132] kvm [13748]:    gpr16: 00000000 1200110c4 120007690 120020f70
[622018.486134] kvm [13748]:    gpr20: 00000000 1201081d0 00000000 fff4c216b0
[622018.486137] kvm [13748]:    gpr24: 00000010 1200110a0 00000ff3 00000003
[622018.486139] kvm [13748]:    gpr28: 120028280 fffbc3c380 fffbc3c4f0 120011148
[622018.486141] kvm [13748]:    hi: 0x00000000
[622018.486143] kvm [13748]:    lo: 0x000001fe
[622018.486145] kvm [13748]:    Status: 0x740000a2, Cause: 0x4000800c
[622018.486147] kvm [13748]:    EPC: 0x1200110a4
[622018.486149] kvm [13748]: guest badvaddr fff69bd068
[622018.486151] kvm [13748]: guest entryhifff69bc000
[622018.486153] kvm [13748]: huxueshi : what's host_cp0_badinstr
[622018.486155] kvm [13748]: update_pc(): New PC: 0x980000fff69b01b0
[622018.486158] kvm [13748]: [0x980000fff69b01b0] HYPCALL 0x16
[622018.486160] kvm [13748]: huxueshi trace leave kvm_vz_vcpu_run:3565

Martins3 commented 3 years ago

Priviledge Table 9.43

Martins3 commented 3 years ago

Insert a probe into musl _Fork:

[635325.574423] kvm [13626]: VCPU Register Dump:
[635325.574425] kvm [13626]:    pc = 0x12000dd24
[635325.574427] kvm [13626]:    exceptions: 00000000
[635325.574430] kvm [13626]:    gpr00: 00000000 20000000 00000000 12002cf00
[635325.574432] kvm [13626]:    gpr04: 000013c0 12001a518 00000000 00000000
[635325.574435] kvm [13626]:    gpr08: 12000dc90 00000000 0000000a 00000001
[635325.574437] kvm [13626]:    gpr12: 00000000 fffda5c000 ffffffffffffffff 00000030
[635325.574441] kvm [13626]:    gpr16: 00000000 12000dcb4 1200070a0 12002c9e0
[635325.574443] kvm [13626]:    gpr20: 00000000 1201081d0 00000000 fff4c21480
[635325.574445] kvm [13626]:    gpr24: 00000010 12000b390 01000000 0500001f
[635325.574448] kvm [13626]:    gpr28: 120033cc0 fffbc8d470 fffbc8d5e0 12000dd18
[635325.574450] kvm [13626]:    hi: 0x00000008
[635325.574452] kvm [13626]:    lo: 0x00000000
[635325.574454] kvm [13626]:    Status: 0x740000a0, Cause: 0x40008008
[635325.574456] kvm [13626]:    EPC: 0x12000dc98
[635325.574457] kvm [13626]: guest badvaddr 12000dc98
[635325.574459] kvm [13626]: guest entryhi12000c000
[635325.574461] kvm [13626]: huxueshi : what's host_cp0_badinstr
[635325.574463] kvm [13626]: update_pc(): New PC: 0x12000dd28
[635325.574466] kvm [13626]: [0x12000dd28] HYPCALL 0x18
[635325.574468] kvm [13626]: huxueshi trace leave kvm_vz_vcpu_run:3565

    if (!ret) {
+    __asm__(".word 0x4200c028");

Martins3 commented 3 years ago

we found the cause register is somewhat unexpected. If the child process reinitialized the cp0, which of course including the cause register, why cause register is changed to 0x40008008?

[x] what's the mechanism of cause register?
[x] what's meaning of 0x40008008 ?
[x] Can I set cause register, check the Linux kernel code?
[x] What's the relation with the stupid five syscall hypercall simulation?
[x] so, I insert another value to cause register?

THE BUG FOUND, HAS NO RELATION WITH CAUSE

Martins3 commented 3 years ago

If writing cause register 0, then it will change to the parent process's case register changed from 0x8 to 0x40008008

I think timer-related code change CAUSE register silently.

Martins3 commented 3 years ago

write_gc0_cause

/*
* Manipulate bits in a guest c0 register.
*/

set_gc0_cause
change_gc0_cause
kvm_read_sw_gc0_cause
kvm_write_sw_gc0_cause
[x] now that every time switch to guest mode, the cause register is used? why change gc0 cause register directly?

if vcpu switch physical cpu, it has to change save and restore it's cause register, that's the reason why kvm_arch_vcpu need miscp_c

Martins3 commented 3 years ago

[20205.975429] kvm [10892]: kvm_arch_vcpu_load: vcpu 000000002b98175c, cpu: 0
[20205.975431] kvm [10892]: change cause _kvm_vz_restore_stable_stimer :  0
[20205.975433] kvm [10892]: save gc0 cause register
[20205.975435] kvm [10892]: change cause kvm_write_sw_gc0_cause 0
[20205.975438] kvm [10892]: kvm_arch_vcpu_load: vcpu 000000002b98175c, cpu: 0
[20205.975440] kvm [10892]: change cause _kvm_vz_restore_stable_stimer :  0
[20205.975442] kvm [10892]: save gc0 cause register
[20205.975444] kvm [10892]: change cause kvm_write_sw_gc0_cause 0
[20205.975447] kvm [10892]: kvm_arch_vcpu_load: vcpu 000000002b98175c, cpu: 0
[20205.975449] kvm [10892]: change cause _kvm_vz_restore_stable_stimer :  0
[20205.975451] kvm [10892]: save gc0 cause register
[20205.975453] kvm [10892]: change cause kvm_write_sw_gc0_cause 0
[20205.975456] kvm [10892]: kvm_arch_vcpu_load: vcpu 000000002b98175c, cpu: 0
[20205.975458] kvm [10892]: change cause _kvm_vz_restore_stable_stimer :  0
[20205.975460] kvm [10892]: save gc0 cause register
[20205.975462] kvm [10892]: change cause kvm_write_sw_gc0_cause 0
[20205.975464] kvm [10892]: kvm_arch_vcpu_load: vcpu 000000002b98175c, cpu: 0
[20205.975466] kvm [10892]: change cause _kvm_vz_restore_stable_stimer :  0
[20205.975469] kvm [10892]: save gc0 cause register
[20205.975471] kvm [10892]: change cause kvm_write_sw_gc0_cause 0
[20205.975474] kvm [10892]: kvm_arch_vcpu_load: vcpu 000000002b98175c, cpu: 0

why child call kvm_arch_vcpu_load so many times ?

[20837.186004] [<ffffffffc09b2a0c>] kvm_vz_vcpu_put+0x124/0x380 [kvm]
[20837.186016] [<ffffffffc09acbc8>] kvm_arch_vcpu_put+0x48/0x68 [kvm]
[20837.186026] [<ffffffffc09943dc>] vcpu_put+0x2c/0x88 [kvm]
[20837.186037] [<ffffffffc09a04d4>] kvm_arch_vcpu_ioctl+0xb4/0x1288 [kvm]
[20837.186048] [<ffffffffc099541c>] kvm_vcpu_ioctl+0x14c/0x830 [kvm]
[20837.186054] [<ffffffff803f71dc>] do_vfs_ioctl+0xc4/0x7c0
[20837.186058] [<ffffffff803f7974>] ksys_ioctl+0x9c/0xe8
[20837.186061] [<ffffffff803f79d4>] sys_ioctl+0x14/0x20

Martins3 commented 3 years ago

[x] whatever takes it's weird to ignore gc0 cause register on vcpu put and load

Martins3 / loongson-dune

ltp: test case #17