GetInstructionSize returns wrong value for four byte sub rsp

SamuelTulach / LightHook

Single-header, minimalistic, cross-platform hook library written in pure C

MIT License

291 stars 46 forks source link

GetInstructionSize returns wrong value for four byte sub rsp #7

Closed ProN00b closed 3 months ago

ProN00b commented 7 months ago

GetInstructionSize returns wrong value for 48 81 ec d0 00 00 00 sub rsp,0xd0 returns 11, should return 7.

VXACDev commented 6 months ago

    else if ((HOOK_R < 4 && (HOOK_C == 5 || HOOK_C == 0xD)) || (HOOK_R == 0xB && HOOK_C >= 8) || (*b == 0xF7 && !(*(b + 1) & 48)) || FindByte(OP1_IMM32, sizeof(OP1_IMM32), *b))

"& 48" ?? I think this should be & 0x48 This is actually in 2 places. I could be wrong, but decimal 48 (0x30) seems... wrong??

SamuelTulach commented 3 months ago

The handling was indeed incorrect. I've added more checks and added a test program. It should now correctly handle instructions with REX.W bit set. Please let me know if you find any more problems.

^ old version would fail at 30+ instances