Closed zhaodice closed 2 months ago
感谢贡献这么多优质代码,由于近期工作较忙,抽不出时间与楼主大大进一步跟进探讨
感谢贡献这么多优质代码,由于近期工作较忙,抽不出时间与楼主大大进一步跟进探讨
谢谢,我已经找到原因了。。。 在汇编中,sp寄存器必须基于STACK_BASE,但是unidbg这边的实现似乎是基于MMAP_BASE(是通过申请内存区域,得到的地址来当sp用)
unicorn/dynarmic对这种行为睁一只眼闭一只眼,但是kvm可能是基于真实CPU的,它忍不了:你这个sp地址有问题啊 问题文件:BaseTask.java 问题代码:
protected final UnidbgPointer allocateStack(Emulator<?> emulator) {
//TODO stackBlock地址基于MMAP_BASE,必须想办法让它基于STACK_BASE
//KVM在使用sp寄存器时会校验,校验失败直接升天
if (stackBlock == null) {
stackBlock = emulator.getMemory().malloc(THREAD_STACK_SIZE, true);
}
return stackBlock.getPointer().share(THREAD_STACK_SIZE, 0);
}
做了个简单的脚本,backend/kvm/src/main/native/driver/build.sh
可以一键编译驱动,当然必须是arm64的linux环境
./build.sh
make ARCH=arm64 CROSS_COMPILE= EXTRA_CFLAGS="-D_ARCH_ARM64_ -I/home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src -I/home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src/arch/arm64 -fno-pic -fno-stack-protector" -C /lib/modules/6.6.20+rpt-rpi-v8/build M=/home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src modules
make[1]: Entering directory '/usr/src/linux-headers-6.6.20+rpt-rpi-v8'
CC [M] /home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src/framework/symbol_resolver_bak.o
LD [M] /home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src/hookFrame.o
MODPOST /home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src/Module.symvers
CC [M] /home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src/hookFrame.mod.o
LD [M] /home/lunzhi/qsign/qsign/kernel/kernel-hook-framework/src/hookFrame.ko
make[1]: Leaving directory '/usr/src/linux-headers-6.6.20+rpt-rpi-v8'
mkdir -p "/home/lunzhi/qsign/qsign/kernel/build"
mkdir -p "/home/lunzhi/qsign/qsign/kernel/build"/kernel-hook
touch "/home/lunzhi/qsign/qsign/kernel/build/Makefile"
make -C /lib/modules/6.6.20+rpt-rpi-v8/build M=/home/lunzhi/qsign/qsign/kernel/build src=/home/lunzhi/qsign/qsign/kernel modules
make[1]: Entering directory '/usr/src/linux-headers-6.6.20+rpt-rpi-v8'
CC [M] /home/lunzhi/qsign/qsign/kernel/build/hcr.o
In file included from /usr/src/linux-headers-6.6.20+rpt-common-rpi/include/asm-generic/bug.h:22,
from /usr/src/linux-headers-6.6.20+rpt-common-rpi/arch/arm64/include/asm/bug.h:26,
from /usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/bug.h:5,
from /usr/src/linux-headers-6.6.20+rpt-common-rpi/arch/arm64/include/asm/cpufeature.h:23,
from /usr/src/linux-headers-6.6.20+rpt-common-rpi/arch/arm64/include/asm/ptrace.h:11,
from /usr/src/linux-headers-6.6.20+rpt-common-rpi/arch/arm64/include/uapi/asm/kvm.h:37,
from /usr/src/linux-headers-6.6.20+rpt-common-rpi/include/uapi/linux/kvm.h:15,
from /home/lunzhi/qsign/qsign/kernel/hcr.c:3:
/home/lunzhi/qsign/qsign/kernel/hcr.c: In function ‘hcr_init’:
/usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/kern_levels.h:5:25: warning: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘kvm_arch_vcpu_ioctl_func’ {aka ‘int (*)(struct file *, unsigned int, long unsigned int)’} [-Wformat=]
5 | #define KERN_SOH "\001" /* ASCII Start Of Header */
| ^~~~~~
/usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/printk.h:427:25: note: in definition of macro ‘printk_index_wrap’
427 | _p_func(_fmt, ##__VA_ARGS__); \
| ^~~~
/home/lunzhi/qsign/qsign/kernel/hcr.c:66:5: note: in expansion of macro ‘printk’
66 | printk(KERN_INFO "hcr: Installed Hook. kvm_arch_vcpu_ioctl=%lx\n",vcpu_ioctl);
| ^~~~~~
/usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/kern_levels.h:14:25: note: in expansion of macro ‘KERN_SOH’
14 | #define KERN_INFO KERN_SOH "6" /* informational */
| ^~~~~~~~
/home/lunzhi/qsign/qsign/kernel/hcr.c:66:12: note: in expansion of macro ‘KERN_INFO’
66 | printk(KERN_INFO "hcr: Installed Hook. kvm_arch_vcpu_ioctl=%lx\n",vcpu_ioctl);
| ^~~~~~~~~
/home/lunzhi/qsign/qsign/kernel/hcr.c: In function ‘hcr_exit’:
/usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/kern_levels.h:5:25: warning: format ‘%lx’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘kvm_arch_vcpu_ioctl_func’ {aka ‘int (*)(struct file *, unsigned int, long unsigned int)’} [-Wformat=]
5 | #define KERN_SOH "\001" /* ASCII Start Of Header */
| ^~~~~~
/usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/printk.h:427:25: note: in definition of macro ‘printk_index_wrap’
427 | _p_func(_fmt, ##__VA_ARGS__); \
| ^~~~
/home/lunzhi/qsign/qsign/kernel/hcr.c:74:5: note: in expansion of macro ‘printk’
74 | printk(KERN_INFO "hcr: Uninstalled Hook. kvm_arch_vcpu_ioctl=%lx\n",vcpu_ioctl);
| ^~~~~~
/usr/src/linux-headers-6.6.20+rpt-common-rpi/include/linux/kern_levels.h:14:25: note: in expansion of macro ‘KERN_SOH’
14 | #define KERN_INFO KERN_SOH "6" /* informational */
| ^~~~~~~~
/home/lunzhi/qsign/qsign/kernel/hcr.c:74:12: note: in expansion of macro ‘KERN_INFO’
74 | printk(KERN_INFO "hcr: Uninstalled Hook. kvm_arch_vcpu_ioctl=%lx\n",vcpu_ioctl);
| ^~~~~~~~~
/home/lunzhi/qsign/qsign/kernel/hcr.c: At top level:
/home/lunzhi/qsign/qsign/kernel/hcr.c:51:12: warning: ‘error_quit’ defined but not used [-Wunused-function]
51 | static int error_quit(const char *msg){
| ^~~~~~~~~~
LD [M] /home/lunzhi/qsign/qsign/kernel/build/hcr-driver.o
MODPOST /home/lunzhi/qsign/qsign/kernel/build/Module.symvers
CC [M] /home/lunzhi/qsign/qsign/kernel/build/hcr-driver.mod.o
LD [M] /home/lunzhi/qsign/qsign/kernel/build/hcr-driver.ko
make[1]: Leaving directory '/usr/src/linux-headers-6.6.20+rpt-rpi-v8'
Build finished, please insmod drivers in build folder ( *.ko ).
total 92
-rw-rw-rw-+ 1 lunzhi lunzhi 8752 Aug 31 16:52 hcr-driver.ko
-rw-rw-rw-+ 1 lunzhi lunzhi 44 Aug 31 16:52 hcr-driver.mod
-rw-rw-rw-+ 1 lunzhi lunzhi 1169 Aug 31 16:52 hcr-driver.mod.c
-rw-rw-rw-+ 1 lunzhi lunzhi 4104 Aug 31 16:52 hcr-driver.mod.o
-rw-rw-rw-+ 1 lunzhi lunzhi 5408 Aug 31 16:52 hcr-driver.o
-rw-rw-rw-+ 1 lunzhi lunzhi 5408 Aug 31 16:52 hcr.o
-rw-rw-rw-+ 1 lunzhi lunzhi 33920 Aug 31 16:52 hookFrame.ko
drwxrwxrwx+ 2 lunzhi lunzhi 4096 Aug 31 16:52 kernel-hook
-rw-rw-rw-+ 1 lunzhi lunzhi 0 Aug 31 16:52 Makefile
-rw-rw-rw-+ 1 lunzhi lunzhi 51 Aug 31 16:52 modules.order
-rw-rw-rw-+ 1 lunzhi lunzhi 0 Aug 31 16:52 Module.symvers
编译后会在当前脚本旁边,创建build文件夹,里面有hookFrame.ko 和 hcr-driver.ko 分别用insmod加载hookFrame.ko和 hcr-driver.ko 即可免PATCH内核使用KVM引擎
ios 下面的 KQueue64Test 跟 KQueueTest 同步你的代码以后运行失败。
ios 下面的 KQueue64Test 跟 KQueueTest 同步你的代码以后运行失败。
好了,KQueue64Test 的问题是因为你在 MachOLoader 莫名其妙把我的栈地址给偏移了,删除掉就行了(因为我不知道你的用意是什么,这个你自己把握) KQueueTest 我发现即使是切到原始分支也会报同样错,所以不是我的问题
C:\Users\yiran\.jdks\jbrsdk-17.0.9\bin\java.exe -javaagent:D:\IntelliJidea\lib\idea_rt.jar=59325:D:\IntelliJidea\bin -Dfile.encoding=UTF-8 -classpath E:\Projects\ProtocolScience\unidbg\unidbg-ios\target\test-classes;E:\Projects\ProtocolScience\unidbg\unidbg-ios\target\classes;E:\Projects\ProtocolScience\unidbg\unidbg-api\target\classes;C:\Users\yiran\.m2\repository\com\github\zhkl0228\unicorn\1.0.14\unicorn-1.0.14.jar;C:\Users\yiran\.m2\repository\org\scijava\native-lib-loader\2.3.5\native-lib-loader-2.3.5.jar;C:\Users\yiran\.m2\repository\com\github\zhkl0228\capstone\3.1.8\capstone-3.1.8.jar;C:\Users\yiran\.m2\repository\net\java\dev\jna\jna\5.10.0\jna-5.10.0.jar;C:\Users\yiran\.m2\repository\com\github\zhkl0228\keystone\0.9.7\keystone-0.9.7.jar;C:\Users\yiran\.m2\repository\commons-codec\commons-codec\1.15\commons-codec-1.15.jar;C:\Users\yiran\.m2\repository\org\apache\commons\commons-collections4\4.4\commons-collections4-4.4.jar;C:\Users\yiran\.m2\repository\commons-io\commons-io\2.11.0\commons-io-2.11.0.jar;C:\Users\yiran\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\yiran\.m2\repository\com\alibaba\fastjson\1.2.83\fastjson-1.2.83.jar;C:\Users\yiran\.m2\repository\com\github\zhkl0228\demumble\1.0.4\demumble-1.0.4.jar;E:\Projects\ProtocolScience\unidbg\backend\dynarmic\target\classes;E:\Projects\ProtocolScience\unidbg\backend\hypervisor\target\classes;E:\Projects\ProtocolScience\unidbg\backend\kvm\target\classes;E:\Projects\ProtocolScience\unidbg\backend\unicorn2\target\classes;C:\Users\yiran\.m2\repository\io\kaitai\kaitai-struct-runtime\0.8\kaitai-struct-runtime-0.8.jar;C:\Users\yiran\.m2\repository\com\googlecode\plist\dd-plist\1.23\dd-plist-1.23.jar;C:\Users\yiran\.m2\repository\org\slf4j\slf4j-api\2.0.13\slf4j-api-2.0.13.jar;C:\Users\yiran\.m2\repository\junit\junit\4.13.2\junit-4.13.2.jar;C:\Users\yiran\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\yiran\.m2\repository\org\slf4j\slf4j-reload4j\2.0.13\slf4j-reload4j-2.0.13.jar;C:\Users\yiran\.m2\repository\ch\qos\reload4j\reload4j\1.2.22\reload4j-1.2.22.jar com.github.unidbg.ios.KQueueTest
unidbg(7780,0x402b518c) malloc: *** error for object 0x40654494: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x406545a4: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x406545b4: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x406545d4: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x406545f4: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x40654614: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x40654634: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x40654654: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
unidbg(7780,0x402b518c) malloc: *** error for object 0x40654674: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
[19:17:07 350] WARN [com.github.unidbg.arm.AbstractARMEmulator] (AbstractARMEmulator$1:67) - Read memory failed: address=0x13, size=1, value=0x0, PC=RX@0x40306134[libobjc.A.dylib]0x2134, LR=RX@0x40305cad[libobjc.A.dylib]0x1cad
MemoryRead8[dynarmic.cpp->MemoryRead8:79]: vaddr=0x13
Process finished with exit code 3
ios 下面的 KQueue64Test 跟 KQueueTest 同步你的代码以后运行失败。
好了,KQueue64Test 的问题是因为你在 MachOLoader 莫名其妙把我的栈地址给偏移了,删除掉就行了(因为我不知道你的用意是什么,这个你自己把握) KQueueTest 我发现即使是切到原始分支也会报同样错,所以不是我的问题
C:\Users\yiran\.jdks\jbrsdk-17.0.9\bin\java.exe -javaagent:D:\IntelliJidea\lib\idea_rt.jar=59325:D:\IntelliJidea\bin -Dfile.encoding=UTF-8 -classpath E:\Projects\ProtocolScience\unidbg\unidbg-ios\target\test-classes;E:\Projects\ProtocolScience\unidbg\unidbg-ios\target\classes;E:\Projects\ProtocolScience\unidbg\unidbg-api\target\classes;C:\Users\yiran\.m2\repository\com\github\zhkl0228\unicorn\1.0.14\unicorn-1.0.14.jar;C:\Users\yiran\.m2\repository\org\scijava\native-lib-loader\2.3.5\native-lib-loader-2.3.5.jar;C:\Users\yiran\.m2\repository\com\github\zhkl0228\capstone\3.1.8\capstone-3.1.8.jar;C:\Users\yiran\.m2\repository\net\java\dev\jna\jna\5.10.0\jna-5.10.0.jar;C:\Users\yiran\.m2\repository\com\github\zhkl0228\keystone\0.9.7\keystone-0.9.7.jar;C:\Users\yiran\.m2\repository\commons-codec\commons-codec\1.15\commons-codec-1.15.jar;C:\Users\yiran\.m2\repository\org\apache\commons\commons-collections4\4.4\commons-collections4-4.4.jar;C:\Users\yiran\.m2\repository\commons-io\commons-io\2.11.0\commons-io-2.11.0.jar;C:\Users\yiran\.m2\repository\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;C:\Users\yiran\.m2\repository\com\alibaba\fastjson\1.2.83\fastjson-1.2.83.jar;C:\Users\yiran\.m2\repository\com\github\zhkl0228\demumble\1.0.4\demumble-1.0.4.jar;E:\Projects\ProtocolScience\unidbg\backend\dynarmic\target\classes;E:\Projects\ProtocolScience\unidbg\backend\hypervisor\target\classes;E:\Projects\ProtocolScience\unidbg\backend\kvm\target\classes;E:\Projects\ProtocolScience\unidbg\backend\unicorn2\target\classes;C:\Users\yiran\.m2\repository\io\kaitai\kaitai-struct-runtime\0.8\kaitai-struct-runtime-0.8.jar;C:\Users\yiran\.m2\repository\com\googlecode\plist\dd-plist\1.23\dd-plist-1.23.jar;C:\Users\yiran\.m2\repository\org\slf4j\slf4j-api\2.0.13\slf4j-api-2.0.13.jar;C:\Users\yiran\.m2\repository\junit\junit\4.13.2\junit-4.13.2.jar;C:\Users\yiran\.m2\repository\org\hamcrest\hamcrest-core\1.3\hamcrest-core-1.3.jar;C:\Users\yiran\.m2\repository\org\slf4j\slf4j-reload4j\2.0.13\slf4j-reload4j-2.0.13.jar;C:\Users\yiran\.m2\repository\ch\qos\reload4j\reload4j\1.2.22\reload4j-1.2.22.jar com.github.unidbg.ios.KQueueTest unidbg(7780,0x402b518c) malloc: *** error for object 0x40654494: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x406545a4: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x406545b4: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x406545d4: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x406545f4: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x40654614: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x40654634: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x40654654: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug unidbg(7780,0x402b518c) malloc: *** error for object 0x40654674: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug [19:17:07 350] WARN [com.github.unidbg.arm.AbstractARMEmulator] (AbstractARMEmulator$1:67) - Read memory failed: address=0x13, size=1, value=0x0, PC=RX@0x40306134[libobjc.A.dylib]0x2134, LR=RX@0x40305cad[libobjc.A.dylib]0x1cad MemoryRead8[dynarmic.cpp->MemoryRead8:79]: vaddr=0x13 Process finished with exit code 3
恢复到 https://github.com/zhkl0228/unidbg/commit/505b92643ba85b5970e0495f37855e0690993f8e 这个版本能正常运行KQueueTest,我的是苹果M1笔记本,使用 dynarmic 后端,windows 上换成 unicorn2 后端试下
前言
暂时没有写一键脚本,所以看起来很长,但它的好处是不再需要重新编译内核了
执行步骤
uname -r
的结果具体原理
此时内核函数kvm_arch_vcpu_ioctl已经被hook,检测到VCPU创建将自动添加HCR_DC 相关实现在 hcr.c
遗留问题
跑代码的时候dmesg输出
一些模拟执行失败
限于能力,这个遗留问题不知道怎么解决,作者大大想进一步了解的话可以继续讨论,不胜感激。