intelxed / xed

The X86 Encoder Decoder (XED), is a software library for encoding and decoding X86 (IA32 and Intel64) instructions
https://intelxed.github.io/
Apache License 2.0
1.39k stars 145 forks source link

VSIB decoding error? #80

Open flobernd opened 6 years ago

flobernd commented 6 years ago

Hey there,

sorry for bothering you again. I tried to decode a random byte sequence and XED reported this:

Attempting to decode: 62 22 f9 85 a2 66 78 5e 24 04 cf 7e 23 d1 17
iclass VSCATTERDPD      category KNC    ISA-extension KNCE      ISA-set KNCE
instruction-length 7
operand-width 64
effective-operand-width 64
effective-address-width 64
stack-address-width 64
iform-enum-name VSCATTERDPD_MEMzv_MASK1mskw_ZMMzf64
iform-enum-name-dispatch (zero based) 0
iclass-max-iform-dispatch 1
Operands
#   TYPE               DETAILS        VIS  RW       OC2 BITS BYTES NELEM ELEMSZ   ELEMTYPE   REGCLASS
#   ====               =======        ===  ==       === ==== ===== ===== ======   ========   ========
0   MEM0           (see below)   EXPLICIT  CW        ZV   64     8     1     64     DOUBLE    INVALID
1   REG1               REG1=K5   EXPLICIT  RW      MSKW   16     2    16      1        INT       MASK
2   REG0            REG0=ZMM28   EXPLICIT   R      ZF64  512    64     8     64     DOUBLE        ZMM
Memory Operands
  0 written BASE= RAX/GPR INDEX= ZMM24/ZMM SCALE= 1 DISPLACEMENT_BYTES= 2 0x00000000000003c0 base10=960 ASZ0=64
  MemopBytes = 8
ATTRIBUTES: KNC_F64 KNC_SCATTER MASKOP_EVEX REQUIRES_ALIGNMENT SCATTER SPECIAL_AGEN_REQUIRED
WRITE-MASKING
ISA SET: [KNCE]

I noticed the modrm.rm field is not 4 (= no SIB/VSIB byte should be present), but XED behaves like there is a valid VSIB. Shouldn't this instruction just #UD?

The correct(?) form disassembles as well, but with different length, mem.index, mem.scale and mem.disp:

Attempting to decode: 62 22 f9 85 a2 64 78 5e 24 04 cf 7e 23 d1 17
iclass VSCATTERDPD      category KNC    ISA-extension KNCE      ISA-set KNCE
instruction-length 8
operand-width 64
effective-operand-width 64
effective-address-width 64
stack-address-width 64
iform-enum-name VSCATTERDPD_MEMzv_MASK1mskw_ZMMzf64
iform-enum-name-dispatch (zero based) 0
iclass-max-iform-dispatch 1
Operands
#   TYPE               DETAILS        VIS  RW       OC2 BITS BYTES NELEM ELEMSZ   ELEMTYPE   REGCLASS
#   ====               =======        ===  ==       === ==== ===== ===== ======   ========   ========
0   MEM0           (see below)   EXPLICIT  CW        ZV   64     8     1     64     DOUBLE    INVALID
1   REG1               REG1=K5   EXPLICIT  RW      MSKW   16     2    16      1        INT       MASK
2   REG0            REG0=ZMM28   EXPLICIT   R      ZF64  512    64     8     64     DOUBLE        ZMM
Memory Operands
  0 written BASE= RAX/GPR INDEX= ZMM31/ZMM SCALE= 2 DISPLACEMENT_BYTES= 2 0x00000000000002f0 base10=752 ASZ0=64
  MemopBytes = 8
ATTRIBUTES: KNC_F64 KNC_SCATTER MASKOP_EVEX REQUIRES_ALIGNMENT SCATTER SPECIAL_AGEN_REQUIRED
WRITE-MASKING
ISA SET: [KNCE]

I only tested the EVEX and MVEX form of VSCATTERDPD and VGATHERDPD, but I'm pretty sure this happens for every instruction that uses the VSIB encoding.


Another (minor) thing I was wondering about: Are KNC instructions only valid in 64-bit mode? Because in the manual it says:

Real-Address Mode and Virtual-8086

UD Instruction not available in these modes

Protected and Compatibility Mode

UD Instruction not available in these modes

for every single KNC VEX/MVEX instruction.

Thanks in advance!

markcharney commented 6 years ago

Can I ask why you care about KNC? The KNC ISA was essentially deprecated with the introduction of AVX512 for Xeon and Xeon Phi. The support in XED for KNC is significantly lower priority.

But yeah, I think I repro'd this for AVX512. I'll look at this tomorrow.

With respect to your second question: sure looks like KNC is 64b mode only. I don't think I enforce that. Not sure if it worth bothering.

flobernd commented 6 years ago

Can I ask why you care about KNC?

KNC is low priority for me as well. Just trying to support as much of the x86-64 isa-extensions as possible. I was simultaneously feeding the MVEX and the EVEX version of XED with random data and the KNC instance was the first one that encountered unexpected behavior (the byte-sequence of my first post in this issue).

I think I repro'd this for AVX512. I'll look at this tomorrow.

Thank you! Seems like a RM=4 besides RM[0b100] in the pattern would do the trick. But you might want to handle this globally in the UISA_VMODRM_xxx()/KNC_VMODRM() functions instead.


With respect to your second question: sure looks like KNC is 64b mode only. I don't think I enforce that. Not sure if it worth bothering.

Yeah, probably not. Was just curious. Thanks for helping me out.

markcharney commented 6 years ago

yeah. the RM=4 is the obvious thing. The other style of constraint is supposed to be equivalent. Suspect there is a little bug lurking somewhere. That is my goal for tomorrow...

markcharney commented 6 years ago

So this is now fixed. But I don't like the fix. I hope I'll have some time over the next few days to do something better. Issue keeps coming up...