capstone-engine / capstone

Capstone disassembly/disassembler framework for ARM, ARM64 (ARMv8), Alpha, BPF, Ethereum VM, HPPA, LoongArch, M68K, M680X, Mips, MOS65XX, PPC, RISC-V(rv32G/rv64G), SH, Sparc, SystemZ, TMS320C64X, TriCore, Webassembly, XCore and X86.
http://www.capstone-engine.org
7.61k stars 1.56k forks source link

[x86] pseudo opcode adjustment for vcmpnlesd is wrong to vcompresspd #2244

Closed tengwu closed 6 months ago

tengwu commented 10 months ago

When disassemble asm code vcmpnlesd %xmm2,%xmm0,%xmm3, whose machine codes are \xc5\xfb\xc2\xda\x06, capstone got a wrong opcode 804(X86_INS_VCOMPRESSPD in x86.h enum x86_insn).

The origin opcode is 797(X86_INS_VCMP in x86.h enum x86_insn), while is adjusted to 804 in cs_disasm function. The displacement(7) is from printSSEAVXCC.

We can reproduce the problem by disassemble \xc5\xfb\xc2\xda\x06 machine codes alonely by capstone. Such as code snippets below:

#include <stdio.h>
#include <stdint.h>
#include "capstone.h"

int main() {
    csh handle;
    cs_insn *insn;
    cs_err err = cs_open(CS_ARCH_X86, CS_MODE_64, &handle);
    const uint8_t code[] = {0xc5, 0xfb, 0xc2, 0xda, 0x06};
    int count = cs_disasm(handle, code, 16, 0, 1, &insn);

    printf("opcode: %d, mnemonic: %s\n", insn[0].id, insn[0].mnemonic);

    cs_free(insn, count);
    cs_close(&handle);

    return 0;
}

Compiling with gcc test.c -I include/capstone/ -L. -lcapstone and run with ./a.out, the output is opcode: 804, mnemonic: vcmpnlesd.